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SUMMARY 


Modern  Aircrew  Training  Devices  {ATDs)  are  equipped  with 
sophisticated  hardware  and  software  capabilities,  known  as  Advanced 
Instructional  Features  (AIFs),  that  permit  a  simulator  instructor  (SI)  to 
prepare  briefings,  manage  training,  vary  task  difficulty/fidelity,  monitor 
performance,  and  provide  feedback  for  flight  simulation  training 
missions.  The  utility  and  utilization  of  the  AIF  capabilities  of  Air 
Force  ATDs  was  explored  by  means  of  a  survey  of  534  Sis  from  Air  Training 
Command,  Military  Airlift  Command,  Strategic  Air  Command,  and  Tactical  Air 
Command  training  sites.  The  primary  purpose  of  the  survey  was  to  provide 
a  database  that  t.ould  be  used  in  defining  the  requirements  for  ATI) 
procurements  and  in  developing  future  ATD  training  programs.  In  general, 
the  features  that  were  rated  highest  were  those  used  for  training 
management,  variation  of  task  difficulty/fidelity,  and  monitoring  student 
performance.  The  level  of  AIF  use  was  affected  somewhat  by  hardware 
and/or  software  unreliability,  implementation  time,  functional 
limitations,  and  design  deficiencies.  However,  the  presumed  training 
value  of  an  AIF  was  the  most  important  determinant  of  its  use. 
Recommendations  are  made  concerning  the  AIF  capabilities  of  future  ATDs 
and  research  aimed  at  determining  the  principles  of  effective  AIF  use. 
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AIRCREW  TRAINING  DEVICES:  UTILITY  AND  UTILIZATION 
UF  ADVANCED  INSTRUCTIONAL  FEATURES 
(PHASE  IV  -  SUMMARY  REPORT) 


I.  INTRODUCTION 

An  Aircrew  Training  Device  (ATD)  is  a  ground-based  substitute 
aircraft  that  permits  student  flight  crews  to  fly  in  a  safe  and  carefully 
controlled  environment.  It  is  frequently  assumed  that  the  training  value 
(i.e.,  utility)  of  an  ATD  is  a  function  of  its  fidelity  or  capability  for 
simulation.  According  to  Adams  (1972),  this  assumption  could  be 
unwarranted. 

I  would  not  consider  the  money  being  spent  on  flight  simulators 
as  staggering  if  we  knew  much  about  their  training  value,  which 
we  do  not.  We  build  flight  simulators  as  realistic  as 
possible,  .  .  .  but  the  approach  is  also  a  cover-up  for  our 
ignorance  about  transfer  because  in  our  doubts  we  have  made 
costly  devices  as  realistic  as  we  can  in  the  hopes  of  gaining 
as  much  transfer  as  we  can.  [In  the  past],  the  users  have  been 
willing  to  pay  the  price,  but  the  result  has  been  an  avoidance 
of  the  more  challenging  question  of  how  the  transfer  might  be 
accomplished  in  other  ways,  or  whether  all  that  complexity 
(i.e.,  fidelity)  is  really  necessary,  (pp.  616-bl 7 ) 

It  is  important  to  realize  that  an  ATD  is  primarily  an  instructional 
device  that  is  designed  to  facilitate  the  acquisition  of  flight  crew 
skills.  Thus,  the  training  value  of  an  ATD  is  determined  not  by  the 
degree  to  which  it  faithfully  simulates  a  particular  aircraft  but  by  the 
way  that  it  is  used  (Caro,  1973).  Yet,  it  appears  that  military  ATDs  are 
more  often  thought  of  as  substitute  aircraft  than  as  instructional  tools. 

A  recent  report  by  the  United  States  General  Accounting  Office  (1983) 
concluded  that  the  armed  services  have  not  sufficiently  analyzed  their 
training  requirements  for  simulators.  Nor  have  they  adequately 
incorporated  simulators  into  their  training  programs.  In  justifying  the 
purchase  of  ATDs,  the  services  have  focused  instead  on  "duplicating  the 
actual  weapon  systems  and  their  surroundings  .  .  .  with  little  reference 
to  how  the  devices  could  meet  training  needs"  (p.  4).  The  GAO  report 
makes  two  recommendations  to  the  Secretary  of  Defense: 

1.  Approve  budget  requests  for  flight  simulators  only  after  the 
services  have  analyzed  their  training  needs  and  proven  that  the 
needs  cannot  be  met  with  existing  simulators. 

2.  Require  the  services  to  incorporate  simulators  into  their 
training  programs,  (p.  7). 

It  is  clear  from  the  report  that  if  the  armed  services  are  to  follow  these 
recommendations,  they  must  better  utilize  and  understand  the  instructional 
capabilities  of  ATDs. 


Advanced  Instructional  Features 


In  order  to  fulfill  its  function  as  an  instructional  device,  an  ATI) 
is  equipped  with  sophisticated  hardware  and  software  capabilities  that 
permit  a  Simulator  Instructor  (SI)  to  brief,  control,  monitor,  and  provide 
feedback  during  simulated  training  missions.  These  capabilities,  some  of 
which  are  listed  in  Table  1,  are  known  as  Advanced  Instructional  Features 
(AIFs).  The  list  was  compiled  from  several  sources,  but  it  was  drawn 
primarily  from  Semple,  Cotton,  and  Sullivan's  (1981)  extensive  report 
describing  the  AIF  capabilities  of  various  military  and  commercial 
devices.  Instructional  features  are  expensive  to  implement,  especially 
those  features  that  require  the  development  of  complex  software.  In  order 
to  justify  these  costs,  some  questions  concerning  the  present  and 
potential  utility  and  utilization  of  AIFs  should  be  answered:  How 
frequently  and  easily  are  AIFs  used?  Are  Sis  adequately  trained  to  use 
AIFs?  Do  AIFs  have  significant  training  value? 

Table  1.  Advanced  Instructional  Features 


BRIEFING  FEATURES 

*  Recorded  Briefing  permits  SI  to  provide  student  with  information  about 

the  simulator  and/or  a  training  mission  through  audiovisual  media 
presentation. 

*  Demonstration  permits  SI  to  demonstrate  optimal  aircrew  performance 

by  means  of  prerecording  and  subsequently  playing  back  segments  of 
simulated  flight. 

*  Instructor  Tutorial  provides  SI  with  self-paced  programmed  instruction 

in  the  capabilities  and  use  of  the  simulator. 

TRAINING  MANAGEMENT  FEATURES 

*  Total  System  Freeze  permits  SI  to  suspend  simulated  flight  by  freezing 

all  system  parameters . 

*  Reset  permits  SI  to  return  the  simulated  aircraft  to  a  stored  set  of 

conditions  and  parameters. 

*  Crash  and/or  Kill  Override  permits  SI  to  allow  simulated  flight  to 

continue  without  interruption  following  a  "crash"  or  "kill." 

*  Automated  Adaptive  Training  is  the  computer-controlled  variation  in  task 

difficulty,  complexity,  and/or  sequence  based  on  student's 
performance. 

*  Programmed  Mission  Scenarios  are  computer-controlled  training  missions 

based  on  pre-programmed  event  sequences. 

*  Manual  Mission  Control  permits  SI  to  modify  programmed  scenarios  during 

a  training  session. 
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Table  1  (Continued) 


VARIATION  OF  TASK  DIFFICULTY/FIDELITY  FEATURES 

*  Automated  Halfunction  Insertion  permits  SI  to  preprogram  a  sequence 

of1  aircraft  component  malfunctions  and/or  emergency  conditions. 

Manual  Halfunction  Insertion  permits  SI  to  modify  preprogrammed 
malfunctions  during  a  training  session. 

*  Environmental  permits  SI  to  vary  environmental  conditions  such  as  wind 

direction  and  velocity,  turbulence,  temperature,  and  visibility. 

Dynamics  permits  SI  to  vary  flight  dynamics  characteristics  such  as 
stability,  system  gain,  cross-coupling,  etc. 

*  Motion  permits  SI  to  provide  student  with  platform  motion  system  cues 

such  as  roll,  pitch,  lateral,  and  vertical. 

*  Partial  Freeze  permits  SI  to  freeze  any  one  or  a  combination  of  flight 

parameters .  Variations  of  this  feature  include: 

Flight  System  Freeze,  which  permits  SI  to  simultaneously  freeze 
flight  control  and  propulsion  systems,  position,  altitude,  and 
heading; 

Position  Freeze,  which  permits  SI  to  simultaneously  freeze  latitude 
and  longitude;  and 

Attitude  Freeze,  which  permits  SI  to  simultaneously  freeze  pitch, 
bank,  and  heading. 

MONITORING  FEATURES 

Closed  Circuit  TV  permits  SI  to  monitor  student’s  behavior  from  the 
instructor  conso 1 e . 

Repeaters/Annunciators  provide  SI  with  replicas  or  analog 
representations  of  flight  instruments  and  controls  at  the 
instructor  console. 

*  Instructor  Console  Displays  permit  SI  to  monitor  parameters  and 

procedures  at  the  instructor  console  by  means  of  alphanumeric 
and/or  graphic  cathode-ray  tube  (CRT)  displays  of  performance  data 
(in  the  survey  this  was  referred  to  as  Parameters  and  Procedures 
Monitoring) . 

Automated  Performance  Alert  provides  SI  with  visual  and/or  auditory 
signals  that  indicate  specific  performance  deficiencies. 
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Table  1  (Concluded) 


FEEDBACK  FEATURES 

*  Record/Playback  permits  SI  to  store  and  subsequently  play  back  a 

segment  of  simulated  flight. 

*  Automated  Performance  Feedback  provides  student  with  visual  and/or 

auditory  signals  (including  verbal  messages)  that  identify 
performance  deficiencies. 

Automated  Voice  Controller  is  the  computer-based  technology  that 
simulates  the  role  of  controller  by  combining  speech  generation, 
speech  recognition,  and  situation  awareness  capabilities. 

*  Hard  Copy  provides  an  alphanumeric  and/or  graphic  record  of  performance 

data. 

*  Performance  Scoring  provides  a  metric(s)  that  summarizes  aircrew  task 

performance  during  a  simulated  mission. 


*  These  features  were  surveyed  during  one  or  more  phases  of  the  project. 


Answers  to  these  questions  have  not  been  fully  provided,  but  relevant 
information  is  available.  One  source  of  information  is  a  series  of 
reports  describing  the  automated  instructional  system  on  the  Advanced 
Simulator  for  Pilot  Training  (ASPT)  at  Williams  AFB,  Arizona  (Faconti  & 
Epps,  1975;  Faconti,  Mortimer,  &  Simpson,  1970;  Fuller,  Waag,  &  Martin, 
1980;  Knoop,  1973).  The  ASPT  is  a  sophisticated  research  device  that 
incorporates  advanced  visual  and  motion  systems,  A-10  and  F-16  cockpits, 
extensive  AIF  capability,  and  an  automated  performance  measurement 
system.  Notwithstanding  the  apparent  training  potential  of  the  ASPT, 

Gray,  Chun,  Warner,  and  Eubanks  (1981)  found  that  Sis  tended  to  use  the 
device  in  a  fairly  conventional  manner.  With  few  exceptions,  the 
instructional  features  were  rarely  used. 

AIF  utility  information  is  available  in  an  important  series  of 
reports  by  R.  G.  Hughes  et  al.  (Bailey  &  Hughes,  1980;  Bailey,  Hughes,  & 
Jones,  1980;  Hughes,  1979;  Hughes,  Hannon,  &  Jones,  1979;  Hughes,  Lintern, 
Wightinan,  Brooks,  &  Singleton,  1982).  The  reports  provide  conceptual 
models  for  AIF-based  simulator  training  programs  and  present  experimental 
evidence  aimed  at  determining  the  training  value  of  particular  features. 

It  is  clear  from  these  reports  that  effective  AIF-based  simulator  training 
is  practicable,  but  systematic  analyses  of  AIF  utility  and  utilization 
patterns  are  required  before  optimal  training  programs,  of  the  kind 
envisioned  by  Knoop  (1973),  can  be  designed: 

The  software  which  will  comprise  flight  simulators  of  the  future 
will  consist  primarily  of  sophisticated  advanced  training  programs 
which  automatically  step  the  student  through  training,  measure  his 
performance  at  each  step,  diagnose  his  problems,  and  alter  the 
difficulty  of  various  tasks  which  are  commensurate  with  his  skill. 

(p.  583) 


A  great  deal  of  information  concerning  AIFs  can  be  found  in  the 
design  guides  of  Caro,  Pohlinan,  and  Isley  (1979)  and  Easter,  Kryway, 

Olson,  Peters,  Slemon,  and  Obermayer  (1986a,  1986b)  and  in  the  Semple  et 
al.  (1981)  report  mentioned  previously.  This  latter  report  is  probably 
the  most  comprehensive  source.  It  was  based  on  interviews  of  ATD 
personnel  at  12  Air  Force,  Navy,  Army,  Coast  Guard,  and  commercial 
training  sites  and  is  one  of  seven  reports  comprising  the  Air  Force 
Simulator  Training  Requirements  and  Effectiveness  Study  (STRES).  The 
report  describes  over  20  AIFs  and  discusses  each  in  terms  of  its 
operation,  related  instructional  features,  instructional  value,  observed 
applications,  utilization  information,  related  research,  and  design 
considerations.  The  interviews  were  "guided"  by  a  checklist  of  topics, 
but  they  were  not  highly  structured.  This  approach  afforded  the 
investigators  flexibility  in  exploring  particular  topics,  but  it  precluded 
systematic  analyses  of  the  data. 

The  Present  Investigation 

The  present  investigation  was  conducted  at  the  request  of  the 
Simulator  System  Program  Office  (SimSPU)  of  the  Air  Force  Systems  Command, 
Aeronautical  Systems  Division  (AFSC/ASD).  The  objectives  of  the 
investigation  were: 

1.  To  document  and  compare  the  utilization  (i.e.,  frequency  and  ease 
of  use)  of  AIFs, 

2.  To  document  and  compare  the  utility  (i.e.,  training  value)  of 

AIFs, 

3.  To  document  and  compare  instructor  training  in  the  use  of  AIFs, 

4.  To  compare  the  utility  and  utilization  patterns  of  AIFs  in 
replacement  (e.g.,  basic,  primary,  lead-in,  initial,  formal,  transition) 
and  continuation  (e.g.,  advanced,  follow-on,  refresher,  operational) 
training  units, 

5.  To  compare  the  utility  and  utilization  patterns  of  AIFs  across 
the  Air  Force  Major  Commands  (MAJCOrls) ,  and 

6.  To  make  recommendations  concerning  the  inclusion  of  AIFs  on 
future  ATDs  and  their  use  in  current  and  future  training  programs. 

Tne  approach  of  the  investigation  consisted  of  surveying  Sis  from  the 
MAJCOMs  in  three  phases  between  March  1982  and  September  1984.  Instructor 
pilots  (IPs)  and  weapon  director  instructors  from  Tactical  Air  Command 
(TAC)  were  surveyed  in  Phase  I.  IPs,  flight  engineers,  and 
radar/navigators  from  Air  Training  Command  (ATC),  Military  Airlift  Command 
(MAC),  and  Strategic  Air  Command  (SAC)  were  surveyed  in  Phase  II. 
Electronic  warfare  instructors,  aerial  gunnery  instructors,  and  weapon 
system  officers  from  ATC,  SAC,  and  TAC  were  surveyed  in  Phase  III.  The 
results  of  those  phases  are  documented  in  three  previous  reports 
(Polzel la,  1983,  1985;  Polzella  &  Hubbard,  1985).  The  present  report 
completes  the  final  phase  of  the  investigation  with  a  summary  of  the 
entire  database. 
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The  purpose  of  this  summary  is  threefold:  (a)  to  extract  findings 
that  are  general izaole  across  MAJCOMs  and/or  levels  of  training,  (b)  to 
determine  differences  that  may  exist  across  MAJCOMs  and/or  levels  of 
training,  and  (c)  to  make  recommendations  for  the  inclusion  of  AIFs  on  new 
ATDs.  The  information  contained  in  this  report  should  be  supplemented 
with  experimental  data,  since  the  conclusions  and  recommendations  are 
derived  from  a  database  consisting  almost  entirely  of  subjective  judgments. 


II.  METHOD 

Subjects 

The  subjects  were  534  simulator-qualified  flight  crew  instructors 
(Sis)  assigned  to  various  ATC  (T-50,  T-51,  T-5),  MAC  (C-5A,  C- 141 ,  CH-3, 
HH-53,  C-130),  SAC  (FB-111A,  T-4,  B-52  weapon  system  trainer  [WST]),  and 
TAC  (F-4E,  F-4G,  F-15,  A-10,  E-3A)  training  sites.  The  subjects  included 
IPs,  weapon  directors  (WDIs),  flight  engineers  (IFEs),  radar/navigators 
(IRNs),  aerial  gunners  (IAGs),  and  electronic  warfare  officers  (EWIs). 

The  distribution  of  particular  Sis  among  the  various  levels  of  training 
and  ATD  sites  that  were  surveyed  in  the  various  phases  are  shown  in 
Appendix  Tables  A-l  through  A-3.  Also  included  are  the  Sis'  mean  (and 
standard  deviation)  number  of  hours  of  instructor  experience.  A  summary 
of  this  information  is  presented  in  Table  2. 

Questionnaires 

The  questionnaires  that  were  used  to  survey  the  instructors  are  shown 
in  Appendix  B,  Phases  I,  II,  and  III.  It  was  necessary  to  construct  a 
separate  version  of  the  questionnaire  for  each  phase  of  the  investigation, 
due  to  differences  in  the  training  requirements  and  capabilities  of  the 
various  ATD  sites.  However,  the  resulting  versions  were  similar  in  most 
respects. 

The  first  page  of  each  questionnaire  requested  information  concerning 
instructor  experience,  a  description  of  a  typical  simulator  training 
session,  and  general  comments  and/or  recommendations.  The  second  page  of 
each  questionnaire  listed  the  features  that  were  to  be  surveyed,  along 
with  their  definitions.  The  list  consisted  of  from  14  to  17  AIFs 
(depending  on  the  version)  drawn  from  Table  1.  A  total  of  19  AIFs  were 
surveyed  during  one  or  more  phases  of  the  project. 

Table  2.  Simulator  Instructors  (Sis)  Surveyed  During  This  Investigation 


Command 

ATD 

Number  of  Sis 

Mean  number  of 
instructor  hours 

ATC 

T-50 

29 

173.6 

T-51 

21 

129.4 

T-5  (EW) 

19 

287.4 

'\v«V,4  t  - 
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Table  2  (Concluded) 


Command 

ATP 

Number  of  Sis 

Mean  number  of 
instructor  hours 

MAC 

C-5A 

46 

490.0 

C-141 

53 

727.7 

C-130 

34 

238.8 

CH-3 

11 

212.0 

HH-53 

17 

381.5 

SAC 

FB-111A 

30 

474.8 

FB-111A  (EW) 

32 

361.0 

T-4  (EW) 

28 

576.3 

B- 52  WST  (EW) 

15 

563.4 

TAC 

F-4E/G 

26 

215.0 

F-4G  (EW) 

32 

95.6 

F  - 1 5 

39 

144.0 

A- 1 0 

38 

21.7 

A-10  (EW) 

33 

75.0 

E-3A  FS 

8 

351.3 

E-3A  MS 

23 

448.1 

On  the  remaining  pages  of  the  survey  were  five  questions  concerning 
the  utility  and  utilization  of  each  feature.  For  Phases  I  and  II,  the 
questions  were  worded  as  follows: 

1.  How  often  have  you  used  each  instructional  feature? 

2.  How  easy  is  it  to  use  each  instructional  feature? 

3.  How  much  training  did  you  receive  in  the  use  of  each 
instructional  feature? 

4.  Rate  the  training  value  of  each  instructional  feature. 

5.  Rate  the  potential  training  value  of  each  instructional  feature. 

The  questions  were  altered  somewhat  for  Phase  III,  at  the  request  of 
TAC/DOT: 


1.  During  five  typical  missions,  how  often  did  you  use  each 
instructional  feature? 

2.  How  difficult/easy  is  it  to  use  each  instructional  feature? 

3.  How  inadequate/adequate  was  the  training  you  received  in  the  use 
of  each  instructional  feature? 

4.  As  presently  implemented  on  your  system,  how  useful  is  each 
instructional  feature? 

5.  Based  on  the  definitions  alone  and  not  your  experience,  how 
potentially  useful  is  each  instructional  feature? 

Except  for  Question  3,  which  asked  Sis  to  rate  the  adequacy  rather 
than  the  amount  of  training  they  received,  the  Phase  III  questionnaire  was 
comparable  to  that  used  for  Phases  I  and  II. 
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For  the  fifth  question.  Sis  were  to  assume  that  they  had  no  prior 
knowledge  of  the  features  and  to  base  their  responses  on  the  feature 
definitions  alone.  This  question  was  included  in  order  to  achieve  a 
common  basis  for  comparison  among  all  Sis.  This  was  not  otherwise 
possible  because  the  various  ATDs  were  not  similarly  equipped. 

Responses  to  each  question  were  indicated  by  checking  the  appropriate 
interval  along  a  7-point,  successive-category  rating  scale.  (On  certain 
questions  a  O-interval  was  included  for  indicating  "not  applicable.")  The 
intervals  of  each  scale  were  labeled  with  descriptive  adjectives,  such  as 
slightly  useful,  fairly  useful,  moderately  useful,  extremely  useful,  and 
indispensable,  in  order  to  facilitate  responding  and  aid  in  interpretation 
of  the  ratings.  Additional  space  was  provided  for  comments. 

Procedure 


The  questionnaire  was  administered  on-site  to  various  sized  (N  =  5  to 
10)  groups  of  Sis.  The  Sis  were  briefed  on  the  purpose  of  the 
investigation  and  copies  of  the  questionnaire  were  distributed  and 
thoroughly  reviewed  prior  to  being  filled  out.  The  questionnaire  could  be 
completed  in  approximately  30  minutes. 


III.  RESULTS 

The  Sis'  responses  to  each  question  were  coded  as  0  (not  applicable) 
to  7  (the  maximum  possible  rating).  The  ratings  were  classified  by  ATD 
(e.g.,  F-4,  F-15,  etc.),  type  of  training  (e.g.,  replacement, 
continuation),  and  AIF  (e.g.,  recorded  briefing,  demonstration,  etc.). 

The  resulting  data  matrix  was  unbalanced  due  to  differences  in  the  number 
of  Sis  and  in  the  AIF  capabilities  of  the  various  ATD  sites.  In  most 
cases,  this  required  that  the  data  from  each  ATD  be  analyzed  separately. 
The  results  of  these  analyses  are  reported  elsewhere  (Polzella,  1983, 
1985;  Polzella  &  Hubbard,  1985).  A  summary  of  the  results  follows. 

General  Trends 


Interrelations  Among  the  Variables 


At  every  training  site,  there  were  clear  interrelations  among  the 
ratings,  as  indicated  by  the  intercorrelations  between  the  ratings  of  each 
feature  across  the  five  questions.  The  observed  Pearson  correlation 
coefficients^  ranged  from  -.05  to  +.80;  95%  of  the  coefficients  were 
positive  and  significant.  Thus,  the  following  generalization  describes 
reasonably  well  the  data  obtained  at  every  site:  A  feature's  rating  on  any 


1  The  Pearson  correlation  coefficient  is  a  statistical  measure  of 
association  that  can  range  from  -1  to  1.  A  value  of  1  represents  a 
perfect  positive  relationship;  and  a  -1,  a  perfect  negative  or  inverse 
relationship.  A  value  of  zero  indicates  no  linear  relation. 
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question  can  be  predicted  with  greater-than-chance  accuracy,  given  its 
rating  on  any  other  question.  For  example,  the  more  useful  a  feature  was, 
the  more  frequently  it  was  used,  the  easier  it  was  to  use,  the  greater  and 
more  adequate  was  the  training  received  in  its  use,  and  the  yreater  was 
its  potential  training  value. 

Multiple  linear  regressions  were  used  to  determine  the  degree  to 
which  the  frequency  of  AIF  use  could  be  predicted  from  the  remaining 
utility  and  utilization  ratings.  Three  potential  predictors  were 
evaluated:  the  ease  of  AIF  use;  the  amount  (or  adequacy)  of  training 
received;  and  AIF  training  value,  a  composite  variable  representing  the 
average  of  the  ratings  on  Questions  4  (training  value)  and  5  (potential 
training  value). 

A  separate  analysis  was  computed  for  each  ATD  site,  and  the  results 
are  summarized  in  Table  3.  Ease  of  use,  training  received,  and  training 
value,  together,  accounted  for  roughly  40%  of  the  variability  in  the 
frequency-of-use  ratings.  The  standardized  regression  coefficients 
(betas)  associated  with  each  variable  are  indications  of  the  strength  of 
that  variable  as  a  predictor;  i.e.,  the  larger  the  beta,  the  greater  the 
predictability.  It  can  be  seen,  for  virtually  every  ATl),  that  the  most 
important  predictor  of  a  feature's  use  was  its  training  value.  For  TAC 
non-EW  ATDs,  ease  of  use,  but  not  amount  of  training  received,  was  also  an 
important  predictor.  For  ATC,  MAC,  and  SAC  non-EW  ATDs,  both  ease  of  use 
and  amount  of  training  received  were  moderately  important  predictors, 
whereas  for  EW  ATDs,  neither  ease  of  use  nor  adequacy  of  training  received 
tended  to  be  important. 

Overview  of  the  Data 


An  overview  of  the  data  can  be  obtained  by  examining  Figures  1 
through  6,  in  which  unweighted  mean  ratings  of  the  frequency  of  use,  ease 
of  use,  training  value,  potential  training  value,  amount  of  training 
received,  and  adequacy  of  training  received  are  shown  for  each  AIF  that 
was  surveyed.  The  unweighted  means  were  calculated  by  averaging  over  the 
means  obtained  at  each  site.  Although  each  unweighted  mean  is  an  unbiased 
average,  unaffected  by  differences  in  the  number  of  Sis  that  were 
surveyed,  it  is  only  representative  of  ATD  sites  at  which  the  particular 
AIF-capabi 1 ity  was  present  and/or  was  surveyed. 

The  features  are  grouped  according  to  function  in  these  figures. 
Briefing  features  are  those  used  for  briefing  the  student  and/or  SI  prior 
to  or  during  a  training  mission.  Training  management  features  permit  the 
SI  to  control  the  structure  and  sequencing  of  tasks  within  a  training 
mission.  Variation  of  task  diff iculty/f idel ity  features  permits  the  SI  to 
control  the  difficulty  of  simulated  missions  through  variations  in  ATD 
fidelity,  configuration,  or  task  load  demands.  Monitoring  features  permit 
the  SI  to  monitor  parameters  (i.e.,  aircraft  states)  and  procedures  (i.e., 
discrete  actions  performed  by  the  student  in  accordance  with  prescribed 
standards)  at  the  instructor  console  in  the  form  of  alphanumeric  and/or 
graphic  cathode-ray  tube  (CKT)  displays  of  performance  data.  Finally, 
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Briefing  Fnfirtt 

Recorded  Briefing - 

Demonstration - 

Training  Management 

Total  System  Freeze - 

Reset  - 

Crash/Kill  Override  - 

Auto  Adaptive  Training  - 

Programmed  Mission  Scenarios  —  - 

Manual  Mission  Control  -  - - 

Variation  of  Task  Difficulty/ Fidelity 

Auto  Malfunction  Insertion  -  -  - 

Environmental  —  - - -  — 

Motion  ------------- 

Partial  Freeze  - -  -  - 

Monitoring  Features 

Parameters  Monitoring  - - 

Procedures  Monitoring  ----- 
Feedback  Features 

Record  Playback  -------- 

Auto  Performance  Feedback  -  -  - 
Hard  Copy  ----------- 

Performance  Scoring  ------ 
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Figure  1.  Freguency  of  AIF  Use.  Mean  ratings  of  esclt  AIF  for  frequency  of  use. 
Circles  represent  RTU  means:  squares.  CTU  means.  Numbers  in 
parentfieses  indicate  the  number  of  ATD  sites  where  the  respective 
AIF  was  rated. 
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Briefing  Features 

Recorded  Briefing  - - - 

Demonstration - - 

Training  Management 

Total  System  Freeze  -  -  -  - 

Reset  ----------- 

Crash  Kill  Override  -  -  -  - 

Auto  Adaptive  Training  -  - 

Programmed  Mission  Scenarios 

Manual  Mission  Control  -  - 

Variation  of  Task  Difficulty  Fidelity 
Auto  Malfunction  insertion 

Environmental  ------ 

Motion  ---------- 

Partial  Freeze  ------ 

Monitoring  Features 

Parameters  Monitoring  -  - 

Procedures  Monitoring  -  - 

feedback  Features 

Record  Playback  -  -  -  -  - 

Auto  Performance  Feedback 

Hard  Copy  -------- 

Performance  Scoring  -  -  - 
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Figure  2  Ease  of  AIF  Use.  Mean  ratings  of  each  AIF  for  ease  of  use.  Circles 
represent  RTU  means:  squares.  CTU  means.  Numbers  in  parentheses 
indicate  the  number  of  ATD  sites  where  the  respecbve  AIF  was  rated. 
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Briefing  Features 

Recorded  Briefing  —  -  —  -  -- 

Demonstration - - 

Training  Management 

Total  System  Freeze  —  -  -  -  -  - 

Reset  -  --  --  —  - - ... 

Crash  Kill  Override  ------- 

Auto  Adaptive  Training  -  -  -  -  - 

Programmed  Mission  Scenarios  -  -  - 

Manual  Mission  Control  -  -  -  -  - 

Variation  of  Task  Difficulty  Fidelity 

Auto  Malfunction  Insertion  -  -  - 

Environmental  --------- 

Motion  ------------- 

Partial  Freeze  . - 

Monitoring  Features 

Parameters  Monitoring  . 

Procedures  Monitoring  ....  - 

feedback  Features 

Record  Playback  . 

Auto  Performance  feedback  *  -  - 
Hard  Copy  ----------- 

’erformance  Scoring  ------ 
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figure  3  Training  Value  of  AIF  Mean  ratings  of  each  AIF  for  training  value. 
Circles  represent  RTU  means,  squares  CTU  means  Numbers  in 
parentheses  indicate  the  number  of  ATD  sites  where  the  respective 
AIF  was  rated 
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Briefing  Features 

Recorded  Briefing - 

Demonstration  - - 

Instructor  Tutorial  -  —  —  -  - 

Training  Management 

Total  System  Freeze  ------ 

Reset  -  -  - - —  -  --  -- 

Crash/Kill  Override  ------ 

Auto  Adaptive  Training  -  -  -  - 

Programmed  Mission  Scenarios  -  - 

Manual  Mission  Control  -  -  -  - 

variation  of  Task  Difficulty  Fidelity 

Auto  Malfunction  Insertion  -  - 

Environmental  -------- 

Motion  ------------ 

Partial  Freeze  - - - 

Monitoring  Features 

Parameters  Monitoring  -  -  -  - 

Procedures  Monitoring  -  -  -  - 

Feedback  Features 

Record  Playback  ------- 

Auto  Performance  Feedback  -  - 

Hard  Copy  ---------- 

Performance  Scoring  -  -  -  -  - 
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CTU<9> 
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CTUI14I 

RTU<5> 
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RTUH7) 

CTU<14) 

RTU<5> 

CTU<4) 


Figure  4.  Potential  Training  Value  for  AIF.  Mean  ratings  of  each  AIF  for  potential 
training  value.  Circles  represent  RTU  means:  squares.  CTU  means. 
Numbers  in  parentheses  indicate  tfie  number  of  ATD  sites  where  the 
res  pecbve  A  IF  was  rated. 
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Briefing  Features 

Recorded  Briefing - 

Demonstration - 

Training  Management 

Total  System  Freeze - -  -  - 

Reset  —  -  -  - - -  -  -  - 

Crash/Kill  Override  ------- 

Auto  Adaptive  Training  - - 

Programmed  Mission  Scenarios  -  -  - 

Manual  Mission  Control  -  -  -  -  - 

Variation  of  Task  Difficulty /Fidelity 

Auto  Malfunction  Insertion  -  -  - 

Environmental  --------- 

Motion  ------------- 

Partial  Freeze  - - 

Monitoring  Features 

Parameters  Monitoring  -  -  -  -  - 

Procedures  Monitoring  -  -  -  -  - 

Feedback  Features 

Record  Playback  -------- 

Auto  Performance  Feedback  -  -  - 
Hard  Copy  ----------- 

Performance  Scoring  ------ 


RTU<2) 

CTU(2) 

RTU<5) 

CTU<5> 

RTU<9> 

CTU<7) 

RTU<11) 

CTU<9> 

RTLK11) 

CTU<9) 

RTUil) 

CTU<1> 

RTU<8) 

CTU<8> 

RTU(O) 
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Figure  S.  Amount  of  Training  Received  in  AIF  Use.  Mean  rabngs  of  each  AIF 
for  the  amount  of  training  received.  Circles  represent  RTU  means: 
squares.  CTU  means.  Numbers  in  parentheses  indicate  the  number  of 
ATD  sites  where  the  respecbve  AIF  was  rated. 
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Briefing  Features 

Recorded  Briefing  --------  RTU<2) 

CTU(1) 

Demonstration  -  —  -  —  -  -  —  RTU<1) 

CTU<1> 

Training  Management 

Total  System  Freeze  -------  RTU<5> 

CTU<4> 

Reset  -  --  --  --  --  --  --  -  RTU<5> 

CTU(4) 

Crash/Kill  Override  -------  RTLHO) 

CTU(O) 

Auto  Adaptive  Training  -----  rtu<0> 

CTU(O) 

Programmed  Mission  Scenarios  -  -  -  rtu<5> 

C  TU<4> 

Manual  Mission  Control  -----  rtu<5> 

CTLH4) 

Variation  of  Task  Difficulty/ Fidelity 

Auto  Malfunction  Insertion  -  -  -  RTU<4> 

CTU<4) 

Environmental  --------  -  RTU(O) 

CTU<0> 

Motion  -  RTU<0) 

CTU(O) 

Partial  Freeze  - RTU<3> 

CTU(3) 

Monitoring  Features 

Parameters  Monitoring  - RTU(4> 

CTU<3> 

Procedures  Monitoring  -----  rtu<4) 

CTUi3) 

feedback  Features 

Record  Playback  --------  rtu<2> 

CTU<2> 

Auto  Performance  Feedback  -  -  -  rtiko> 

CTU<0) 

Hard  Copy  . - .  RTU<3> 

CTU<2) 

Performance  Scoring  ------  rtu<3> 

CTU(2) 


Figure  6.  Adequacy  of  Training  Received  in  AIF  Use.  Mean  ratings  of  each 
AIF  for  adequacy  of  training  received.  Circles  indicate  the  number 
of  ATD  sites  where  the  respective  AIF  was  rated. 


feedback  features  permit  the  SI  to  provide  the  student  with  various  forms 
of  performance  feedback. 

Two  general  trends  are  revealed  in  the  figures.  First,  training 
management,  variation  of  task  difficulty/fidelity,  and  monitoring  features 
tended  to  receive  the  highest  ratings,  whereas  briefing  and  feedback 
features  tended  to  receive  the  lowest  ratings.  This  trend  was  observed  at 
every  ATD  site. 

The  second  general  trend  revealed  in  the  figures  is  that  there  were 
differences  in  the  ratings  of  features  across  the  two  levels  of  training. 
The  figures  indicate  that  certain  features  tended  to  receive  higher 
ratings  from  replacement  training  unit  ( KTU )  Sis  (e.g.,  performance 
scoring),  whereas  others  tended  to  receive  higher  ratings  from 
continuation  training  unit  { CTU )  Sis  (e.g.,  programmed  mission 
scenarios).  However,  there  were  many  exceptions  to  this  trend,  which  are 
noted  in  the  sections  that  follow. 

Utility  and  Utilization  Ratings  of  Each  AIF 

A  more  detailed  summary  of  the  data  is  presented  in  the  tables  that 
follow.  The  tables  list  means  and  standard  deviations  of  the  Sis’  ratings 
of  the  frequency  of  use,  ease  of  use,  amount  of  training  received, 
adequacy  of  training  received,  training  value,  and  potential  training 
value  for  each  of  the  19  AIFs  that  were  surveyed,  respectively.  The  data 
are  tabulated  according  to  MAJCOM,  and  statistics  are  listed  separately 
for  the  two  levels  of  training  and  for  every  ATD  having  the  particular  AIF 
capability  (assuming  that  it  was  included  on  the  given  questionnaire). 

ATC  ATDs  include  T-50,  T-51,  and  T-5  (electronic  warfare,  EW),  RTU  only. 
MAC  ATDs  include  C-5/C-141,  C-130,  and  CH-3/HH-53,  RTU  and  CTU.  SAC  ATDs 
include  FB-111A,  FB-111A  (EW),  T-4  (EW),  and  B-52  WST  (EW),  RTU  and  CTU. 
TAC  ATDs  include  F-4E/F-4G,  F-4G  (EW),  F-15,  A-10,  A-10  (EW),  E-3A  Flight 
Simulator,  and  E-3A  Mission  Simulator,  RTU  and  CTU.  Note  that  the  data 
from  some  of  the  training  sites  were  combined  (e.g.,  C-5/C- 1 41 , 

CH-3/HH-53,  and  F-4E/F-4G).  It  seemed  appropriate  to  combine  these  data 
since  the  respective  training  missions  were  highly  similar  and  the 
comparable  mean  ratings  were  nearly  identical. 

Briefing  Features 


Recorded  Briefing  (Table  4) 

Incidence.  Recorded  briefing  was  surveyed  at  every  ATD  site;  it  was 
available  for  use  on  only  four  devices:  T-5,  T-4,  A-10,  and  E-3A  mission 
simulator. 

Utility  and  utilization.  Except  for  T-5  Sis,  who  regularly  used 
recorded  briefing,  the  majority  of  instructors  never  used  the  feature;  and 
several  were  not  even  aware  that  the  capability  existed.  Instructors' 
written  comments  suggested  a  preference  for  informal  briefings,  which 
could  be  auapted  to  the  particular  needs  of  individual  students  and 
instructors. 
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Comparisons  across  MAJCOMs  and  levels  of  training.  The  potential 
utility  of  recorded  briefing  tended  to  be  highest  for  ATC  ATDs  and  lowest 
for  TAC  ATDs.  For  six  of  the  seven  TAC  ATDs  (all  except  the  F-4E/F-4G), 
the  RTU  Sis'  potential  training  value  ratings  were  higher  than  those  of 
the  CTU  Sis.  (This  difference  was  significant  for  the  E-3A  flight 
simulator.)  In  contrast,  for  three  of  the  four  SAC  ATDs  (all  except  the 
FB-111A),  the  CTU  Sis'  potential  training  value  ratings  were  higher  than 
those  of  the  RTU  Sis. 

Demonstration  (Table  5) 

Incidence.  Demonstration  was  surveyed  at  every  ATD  site.  It  was 
available  for  use  on  six  devices:  C-130,  CH-3/HH-53,  FB-111A,  A-10,  A-10 
(EW),  and  E-3A  mission  simulator. 

Utility  and  utilization.  The  ratings  of  the  demonstration  feature 
were  among  the  lowest  ratings  given  for  any  feature.  There  were  two  major 
complaints  at  each  site:  First,  implementing  the  feature  was 
time-consuining  and  often  unreliable.  Second,  an  enormous  effort  was 
required  to  update  and  maintain  current  scenarios  through  software 
development,  which  resulted  in  an  insufficient  number  of  demos  to  meet 
training  requirements. 

Comparisons  across  MAJCOMs  and  levels  of  training.  There  were  no 
significant  differences  in  utility  and  utilization  across  MAJCOMs.  The 
E-3A  flight  simulator  RTU  Sis  rated  demonstration  higher  in  potential 
training  value  than  did  the  CTU  Sis;  however,  there  were  no  other 
significant  differences  across  the  two  levels  of  training. 

Instructor  Tutorial  (Table  6) 

Incidence.  Instructor  tutorial  was  surveyed  at  every  ATu  site; 
however,  none  of  the  ATDs  included  this  capability.  This  feature  differed 
from  the  other  AIFs  surveyed  in  that  its  purpose  was  the  instruction  of 
Sis  in  the  operation  of  the  ATD. 

Utility  and  utilization.  The  potential  training  value  ratings  were 
all  in  the  moderate  range  of  the  scale.  The  instructors'  written  comments 
suggested  that  they  prefer  "hands-on"  experience  and/or  "face-to-face" 
tutorials  on  the  operation  of  the  instructor's  console. 

Comparison  across  MAJCOMs  and  levels  of  training.  The  potential 
training  ratings  were  statistically  equivalent  across  the  MAJCOMs  and 
between  the  two  levels  of  training. 

Training  Management  Features 


Total  System  Freeze  and  Reset  (Tables  7  and  8) 


Incidence.  Total  system  freeze  and  reset  were  surveyed  at  every  ATU 
site.  Fotal  system  freeze  was  available  for  use  on  every  device  except 
the  F-4E/F-4G,  F-15,  and  E-3A  flight  simulator  (but  it  was  present  on  the 
E-3A  mission  simulator).  Reset  was  available  on  every  device  except 
the  E-3A  mission  simulator. 


Mean  Ratings  (and  Standard  Deviations)  of  the  Frequency  of  Use, 

Ease  of  Use,  Amount  of  Training  Received,  Adequacy  of  Training  Received, 

Training  Value,  and  Potential  Training  Value  by  RTU  (R)  and  CTU  (C)  Simulator  Instructors 
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tings  (and  Standard  Deviations)  of  the  Frequency  of  Use, 

,  Amount  of  Training  Received,  Adequacy  of  Training  Received, 

Potential  Training  Value  by  RTU  (R)  and  CTU  (C)  Simulator  Instructors 


Utility  and  utilization.  It  makes  sense  to  consider  total  system 
freeze  and  reset  together,  since  they  were  so  often  used  in  conjunction. 
Sis  used  total  system  freeze  when  they  wished  to  temporarily  suspend  the 
training  mission  in  order  to  provide  the  student  with  instruction  or 
feedback.  They  then  used  reset  in  order  to  restore  the  mission.  There 
were  few  problems  associated  with  the  use  of  these  features.  The 
frequency-  and  ease-of-use  ratings  indicated  that  these  features  tenaed  to 
be  implemented  very  easily  and  with  moderate  regularity.  The  only 
exceptions  occurred  at  sites  in  which  (a)  it  was  necessary  to  reinitialize 
the  ATD  in  order  to  resume  training  following  the  use  of  total  system 
freeze  (e.g.,  E-3A  mission  simulator),  or  (b)  the  use  of  reset  was  not 
sufficient  to  permit  resumption  of  the  mission  from  the  point  at  which  it 
had  been  suspended  (e.g.,  F-15  simulator). 

Comparisons  across  MAJCQMs  and  levels  of  training.  The  EW  Sis' 
utility  ratings  of  total  system  freeze  and  reset  were  high,  and  did  not 
differ  across  the  MAJCOHs.  There  was  more  variability  in  the  non-EW  Sis' 
ratings;  utility  was  highest  for  AT C  Sis,  lowest  for  TAC  Sis. 

On  several  ATUs,  there  were  significant  differences  in  utility  and 
utilization  across  the  two  levels  of  training.  Most  of  these  differences 
showed  higher  ratings  by  RTU  Sis.  On  the  F-15  ATD,  the  RTU  Sis  used  reset 
more  frequently  than  did  the  CTU  Sis.  They  also  received  more  training  in 
the  use  of  reset  and  rated  the  feature  as  having  greater  training  value. 

On  the  A-10  ATD,  the  RTU  Sis  used  total  system  freeze  more  often  than  did 
the  CTU  Sis;  and  on  the  E-3A  flight  simulator,  the  RTU  Sis  rated  the 
feature  higher  in  potential  training  value  than  did  the  CTU  Sis.  However, 
on  the  C— 5/C- 141  ATD,  the  CTU  Sis  assigned  higher  training  value  ratings 
to  total  system  freeze  than  did  the  RTU  Sis. 

Crash/Kill  Override  (Table  9) 

Incidence.  Crash/kill  override  was  surveyed  at  every  non-EW  ATD 
site.  It  was  available  for  use  on  all  non-EW  devices. 

Utility  and  utilization.  We  can  account  for  the  generally  high 
ratings  by  considering  that  although  crash/kill  override  is  more  properly 
viewed  as  a  variation  of  the  task  difficulty  feature,  it  was  more  often 
used  for  training  management.  If  "crashes"  or  "kills"  were  permitted  to 
occur,  a  tedious  reinitialization  of  the  ATD  would  typically  be  required. 
Thus,  the  feature  was  used,  more  often  than  not,  in  order  to  avoid  the 
loss  of  instruction  time. 

Comparisons  across  MAJCQMs  and  levels  of  training.  "Crashes"  were 
most  often  permitted  (i.e.,  crash/kill  override  was  not  activated)  on  the 
ATC  ATDs  (viz.,  T-50,  T-51)  and  on  TAC's  E-3A  mission  simulator.  On  the 
C-5/C-141  ATD,  the  RTU  Sis  used  the  feature  more  often  than  did  the  CTU 
Sis;  and  on  the  F-15  ATD,  the  RTU  Sis  rated  the  feature  as  easier  to  use 
than  did  the  CTU  Sis. 
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Training  5.0  5.8  4.8  5.5  5.2 

Value-C  (1.7)  (0.8)  (1.8)  (1.3)  (1.8) 


Automated  Adaptive  Training  (Table  10) 

Incidence.  Automated  adaptive  training  was  surveyed  at  every  non-EW 
ATD  site;  it  was  available  for  use  only  on  the  F-4E  ATO. 

Utility  and  utilization.  Utilization  of  automated  adaptive  training 
was  minimal,  the  P-4E  Sis'  ratings  of  the  frequency  of  use,  ease  of  use, 
and  amount  of  training  received  were  lower  than  95*  of  the  comparable 
ratings  of  the  other  training  management  features.  According  to  the  Sis 
who  used  automated  adaptive  training  (the  majority  did  not  use  it  and  more 
than  20*  were  unaware  of  its  availability),  there  were  two  major  problems 
with  the  feature:  The  first  was  that  “it  takes  the  instructor  out  of  the 
loop";  the  second  was  that  it  could  be  applied  only  at  certain  points 
during  a  mission. 

Comparisons  across  the  MAJCOHs  and  levels  of  training.  The  potential 
training  value  ratings  were  statistically  equivalent  across  the  MAJCOMs 
and  between  the  two  levels  of  training,  with  one  exception:  The  E-3A 
flight  simulator  RTU  Sis'  mean  rating  was  significantly  higher  than  that 
of  the  CTU  Sis. 

Programmed  Mission  Scenarios  and  Manual  Mission  Control  (Tables  11  and  12) 

Incidence.  Programmed  mission  scenarios  were  surveyed  at  every  ATD 
site,  they  were  available  for  use  on  all  but  the  T-50,  T-51,  C-5/C-141, 
and  CH-3/HH-53  ATDs.  Manual  mission  control  was  surveyed  during  the  EW 
phase  of  the  project.  Data  were  collected  from  each  of  the  EW  trainers 
except  the  FB-111A. 

Utility  and  utilization.  Sis'  ratings  of  the  utility  and  utilization 
of  programmed  mission  scenarios  varied  greatly  across  training  devices. 

For  example,  in  non-EW  applications,  frequency  of  use  ranged  from  1.0 
(A-10  ATD,  RTU)  to  6.0  (E-3A  flight  simulator,  CTU).  In  general,  the  most 
favorable  ratings  were  obtained  from  the  T-5,  T-4,  and  B-52  WST  Sis. 

There  were  two  important  factors  that  limited  the  use  of  programmed 
mission  scenarios.  First,  a  typical  simulated  mission  consisted  of  a  long 
and  complex  sequence  of  events,  and  the  programming  of  scenarios  was  thus 
a  tedious  and  difficult  task.  Not  surprisingly,  there  was  an  insufficient 
number  of  scenarios  to  accomplish  training  at  most  sites.  Those  that  were 
available  were  frequently  characterized  as  "unreliable,"  "limited,"  or 
"outdated."  Second,  approximately  25*  of  the  C-130,  F-4E/F-4G,  F-15,  and 
A-10  Sis  commented  that  they  preferred  the  increase  in  instructional 
flexibility  afforded  by  manual  mission  control.  Unfortunately,  ratings  of 
manual  mission  control  were  only  obtained  from  EW  Sis,  and  these  ratings 
closely  matched  those  of  programmed  mission  scenarios,  with  one  exception: 
T-5  Sis  greatly  preferred  programmed  to  manual  scenarios. 
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Table  12.  Manual  Mission  Control 

Mean  Ratings  (and  Standard  Deviations)  of  the  Frequency  of  Use, 

Ease  of  Use,  Amount  of  Training  Received,  Adequacy  of  Training  Received, 

Training  Value,  and  Potential  Training  Value  by  RTU  (R)  and  CTU  (C)  Simulator  Instructors 


Comparisons  across  MAJCOMs  and  levels  of  training.  Although  there 
were  no  obvious  differences  in  the  ratings  of  programmed  mission  scenarios 
across  MAJCOMs,  the  feature  appeared  to  be  more  important  for  CTU 
training.  Thus,  F8-111A,  FB-111A  (EW),  F-J5,  and  E-3A  flight  simulator 
CTU  Sis  used  the  feature  significantly  more  often  than  did  their  RTU 
counterparts.  Moreover,  A-10  CTU  Sis  rated  programmed  mission  scenarios 
higher  in  training  value  than  did  rt-10  RTU  Sis.  The  ratings  of  manual 
mission  control  did  not  differ  across  the  two  levels  of  training. 
Comparisons  across  MAJCOMs  were  precluded  due  to  a  lack  of  data. 

Variation  of  Task  Difficulty/Fidelity  Features 
Automated  Malfunction  Insertion  (Table  13) 

Incidence.  Automated  malfunction  insertion  was  surveyed  at  every  ATI) 
site.  It  was  available  for  use  on  every  device  except  the  C-5/C-141, 
CH-3/HH-53,  and  FB-111A  ATDs,  and  the  E-3A  mission  simulator. 

Utility  and  utilization.  The  ratings  of  this  feature  varied 
greatly^  On  two  devices  (Viz.,  E-3A  flight  simulator,  B-52  WST)  it  worked 
well,  and  the  frequency  of  use  was  higher  than  it  was  on  the  remaining 
devices.  For  most  of  those  ATDs,  especially  the  F-4E/F-4G,  F-15,  and 
A-10,  it  was  said  that  it  was  time-consuming  to  implement  and  unreliable, 
and  did  not  always  reflect  mission  profiles.  Moreover,  almost  20%  of  the 
Sis  commented  that  they  preferred  to  insert  malfunctions  manually.  There 
was  a  clear  parallel  between  the  utility  and  utilization  of  automated 
malfunction  insertion  and  that  of  programmed  mission  scenarios.  The 
operational  difficulties  and  limitations  of  both  features  restricted  their 
use,  and  a  substantial  number  of  Sis  preferred  the  benefits  of  manual 
malfunction  insertion  and  manual  mission  control,  respectively. 

Comparisons  across  MAJCOMs  and  levels  of  training.  Table  13  clearly 
suggests  that  the  utility  and  utilization  of  automated  malfunction 
insertion  was  lowest  for  TAC  fighter  ATDs.  The  only  significant 
comparisons  across  the  two  levels  of  training  were  for  the  C- 130  ATI).  The 
feature  was  used  more  often  and  was  rated  higher  in  training  value  by  RTU 
than  by  CTU  Sis. 

Environmental  (Table  14) 

Incidence.  Environmental  was  surveyed  at  every  non-EW  ATD  site.  The 
capabi Hty  was  available  for  use  on  every  device  except  the  FB-111A  ATD. 

Utility  and  utilization.  The  mean  ratings  of  the  environmental 
feature  were  uniformly  in  the  moderate  to  high  range  of  the  scales,  with 
one  exception:  the  E-3A  mission  simulator.  The  favorable  ratings  of  this 
feature  were  apparently  due  to  its  easy,  reliable  operation  and  its  value 
in  training  instrument  flying  under  adverse  weather  conditions.  Frequency 
of  use  was  significantly  lower  on  the  E-3A  mission  simulator. 

Environmental  simulation  on  this  device  was  limited  to  "winds  aloft." 
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Table  13.  Automated  Malfunction  Insertion 
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Comparison  across  MAJCOMs  and  levels  of  training.  There  were  no 
significant  differences  in  the  utility  and  utilization  of  the 
environmental  feature  across  MAJCOMs.  However,  there  were  several 
significant  differences  across  the  two  levels  of  training.  A-10  RTU  Sis 
used  it  more  often  and  rated  it  higher  in  training  value  than  did  A-10  CTU 
Sis;  and  F - 1 5  RTU  Sis  rated  it  easier  to  use  and  received  more  training  in 
its  use  than  did  F-15  CTU  Sis. 

Motion  (Table  15) 

Incidence.  Platform  motion  was  the  only  form  of  motion  cueing 
surveyed.  It  was  surveyed  at  every  non-EW  ATD  site.  It  was  available  for 
use  on  the  T-50,  T-51,  C-5/C-141,  C-130,  CH-3/HH-53,  FB-111A,  and  F-15 
ATDs,  and  the  E-3A  flight  simulator. 

Utility  and  utilization.  Except  for  the  F-15  ATD,  the  utility  and 
utilization  ratings  of  operational  platform  motion  cueing  systems  were 
uniformly  in  the  high  range  of  the  rating  scales.  Over  60%  of  the  F-15 
Sis  commented  that  the  motion  simulation  was  unrealistic  and  would  not 
yield  positive  transfer  of  training  to  aircraft  itself. 

Comparisons  across  MAJCOMs  and  levels  of  training.  In  general ,  the 
utility  (actual  and  potential)  and  utilization  ratings  of  ATC,  MAC,  and 
SAC  motion  systems  were  clearly  higher  than  those  of  TAC.  A  more  accurate 
statement  would  be  that  non-fighter  aircraft  ATD  platform  motion  systems 
were  evaluated  more  favorably  than  were  fighter  aircraft  ATD  platform 
motion  systems,  since  TAC's  E-3A  flight  simulator  motion  system  received 
extremely  high  ratings. 

There  were  several  significant  differences  in  the  ratings  across  the 
two  levels  of  training.  The  F-15  KTU  Sis  rated  motion  higher  in  training 
value  and  potential  training  value  than  did  the  F-15  CTU  Sis,  whereas  the 
C-5/C-141  and  A-10  CTU  Sis  rated  motion  higher  in  training  value  and 
potential  training  value,  respectively,  than  did  the  RTU  Sis. 

Partial  Freeze  (Table  16) 

Incidence.  Partial  freeze  was  surveyed  at  every  ATD  site.  It  was 
available  for  use  on  every  device  except  the  T-5  and  T-4  ATDs,  and  the 
E-3A  mission  simulator. 


Utility  and  utilization.  As  with  total  system  freeze,  the  ratings  of 
partial  freeze  indicated  that  it  tended  to  be  implemented  very  easily  and 
with  moderate  regularity.  Although  it  is  properly  considered  a  variation 
of  the  task  difficulty  feature,  partial  freeze  was  used  in  a  manner 
similar  to  crash/kill  override  (see  above);  that  is,  to  manage  training. 

In  fact,  over  80%  of  Sis  occasionally  used  partial  freeze  as  a  substitute 
for  total  system  freeze  in  order  to  temporarily  suspend  the  training 
session  and  instruct  the  student.  Partial  freeze  appeared  to  offer  two 
advantages  over  total  system  freeze  for  this  purpose.  First,  on  certain 
devices  (e.g.,  C-5/C-141,  FB-lllA)  it  was  simply  easier  to  reinitialize 
the  ATD  following  a  partial  rather  than  total  freeze.  Second,  by  freezing 
only  particular  flight  parameters,  Sis  were  able  to  instruct  while  still 


a 


<C  <cj*  ro 

n  oo  *  • 

t  U,  MO  — 


o 

o  — 

<  t 

k~  < 


-O  X)  "O  CM 
1-0  •—  CM  •“ 


< 

<  —  — 

oo  —  3 


•  T3 

LT> 

—  > 

4/  , 
< v  a  - 


CO  CO  MO 
ID  —  M3  »— 


CVJ  CM  -O  o 

D  ■ —  CM 


1 


ro  -y\  cm  o 
vo  o  vo  <~- 


CM  ^  • — 

M3  —  vjO  *— 


CO  •—  NO 
ID  < —  VO  •— 


CM  CO  Cs. 

M3  —  M3  O 


3-  —  O 

ID  • —  .  • 

II  lD  CM 

(_>  V_J  — ■ - 


r>.  O  Oi  co 
LD  ■ —  ID  ♦— 


^  CO  —  — 
tO  —  M3  — 


_ _ _ 

_ _ _ 

, — 

CM  <y» 

r-» 

00  o 

o  o 

a>  •— 

LD 

1 

M3  — 

<M>  O 

rt  CM 

M3  »“ 

ID  r— 

►— 

'**“ ’ 

’*■**’ 

O 

M>  CM 

rf  O 

VO  MO 

—  o 

CO  #— 

D 

1 

M3  — 

M3  O 

— ' 

MO  — 

VO  r- 

►— 

— * 

— " 

w 

M- 

M- 

ae 

i_j 

o  oc 

O  CJ 

>> 

>v 

M-  1 

i 

1 

1 

•— 

u  a: 

U  -3 

cc 

<-> 

o  cr>  to 

O  3»T> 

>*  a>  to 

>>  cn  T3 

o» 

Oi 

•n  cc 

O  CTI 

C  1 

C  1 

1 

1 

c  <u 

C  QJ 

U  C  1> 

L'  C  D 

c  ac 

C  O 

-  c  a 

-  cu 

Qj  QJ 

<L»  V 

OJ 

OJ 

*-»•»-> 

*-»  p-  > 

<TJ  -  > 

•0  -  > 

■i-  i 

■*“  1 

*-»  r-  | 

I 

3  «/3 

3  i/3 

l/l 

l/l 

c  c  •*- 

c:  c  •»- 

3  C  *— 

3  C  - 

c  V 

C  0) 

C  C  QJ 

C  C  0) 

cr=» 

cr  r> 

o>  zo 

<u  n> 

3  0> 

3  *“  OJ 

O’-'-  01 

a***-  ai 

■—  3 

3 

a»  a 

0J  ■*-  3 

OJ 

<v 

i/i 

l/l 

Q  «  U 

O  fl  u 

0MJ  U 

ai  t>  u 

<0  r- 

"9  — 

*-»  m  »— 

*->  "9  r— 

i_  i*_ 

u  u- 

<9  **- 

-T3  M- 

E  L  0) 

5  i.  <u 

T3  U  OJ 

■o  i-  a> 

u.  m 

t  « 

O  U  <Q 

O  fc.  10 

u.  o 

u.  o 

UJ  O 

LbJ  O 

•x y-  oc 

<  »-  QC 

«t  ♦—  CSC 

«c  *-  oc 

f—  >> 

Q-  »—  >• 

Cl.  ^ 

"*A 


Mean  Ratings  (and  Standard  Deviations)  of  the  Frequency  of  Use, 

Ease  of  Use,  Amount  of  Training  Received,  Adequacy  of  Training  Received, 

Training  Value,  and  Potential  Training  Value  by  RTU  (R)  and  CTU  (C)  Simulator  Instructors 
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maintaining  a  relatively  realistic  simulated  environment.  In  contrast,  on 
the  A- 10  ATD  and  the  B-5Z  WST,  it  was  more  time-consuming  to  implement  a 
partial  than  a  total  system  freeze,  and  partial  freeze  was  used 
significantly  less  often  on  these  devices. 

Comparisons  across  MAJCOMs  and  levels  of  training.  The  utility  and 
utilization  of  partial  freeze  did  not  differ  across  the  MAJCOMs.  On  one 
device,  the  F-15  simulator,  there  were  clear  differences  across  the  two 
levels  of  training:  F-15  RTU  Sis  used  partial  freeze  more  often,  found  it 
easier  to  use,  received  more  training  in  its  use,  and  rated  it  higher  in 
training  value  than  did  F-15  CTU  Sis.  However,  the  CTU  Sis  assigned 
higher  potential  training  value  ratings  to  total  system  freeze  than  did 
the  RTU  Sis.  On  the  C-5A/C-141,  partial .freeze  was  also  used  more  often 
by  RTU  than  by  CTU  Sis. 


Monitoring  Features 

Parameters  and  Procedures  Monitoring  (Tables  17  and  18) 

Incidence.  Parameters  and  procedures  monitoring  were  surveyed  on  all 
devices  except  TAC's  non-EW  ATDs.  Parameters  monitoring  was  available  for 
use  on  the  T-5,  C-130,  CH-3/HH-53,  FB-111A,  T-4,  B-52  WST,  and  A-10  (EW) 
ATDs.  Procedures  monitoring  was  available  on  the  same  devices,  with  the 
exception  of  the  CH-3/HH-53  ATD. 

Utility  and  utilization.  The  utility  and  utilization  of  these 
features  was  very  high.  Over  90%  of  the  means  were  in  at  least  the 
moderate  range  of  the  rating  scales.  The  ratings  tended  to  be  highest  for 
those  devices  that  required  Sis  to  monitor  performance  from  a  remote 
console  (e.g.,  FB-111A,  T-4),  whereas  the  ratings  were  significantly  lower 
for  ATDs  in  which  the  instructor  console  was  located  in  the  simulation 
chamber  with  the  student  (e.g.,  C-130,  CH-3/HH-53).  Under  these 
circumstances,  the  majority  of  Sis  preferred  to  monitor  student 
performance  "over-the-shoulder"  by  looking  at  the  instruments  and  switches 
directly.  One  exception  to  this  trend  was  the  A-10  ATD  (EW),  for  which 
parameters  and  procedures  monitoring  received  low  ratings  despite  the 
remote  location  of  these  features. 

Comparisons  across  MAJCOMs  and  levels  of  training.  The  utility  and 
utilization  of  parameters  and  procedures  monitoring  did  not  differ 
significantly  across  the  MAJCOMs.  There  were  significant  differences 
across  the  two  levels  of  training  on  only  one  device,  the  A-10  ATD  (EW). 
The  A-10  (EW)  RTU  Sis  used  parameters  and  procedures  monitoring  more  often 
than  did  the  A-10  (EW)  CTU  Sis.  In  addition,  the  A-10  (EW)  RTU  Sis  rated 
parameters  monitoring  higher  in  training  value  than  did  the  A-10  (EW)  CTU 
Sis. 


Feedback  Features 


Record/Playback  (Table  19) 

Incidence.  Record/playback  was  surveyed  at  every  ATD  site.  It  was 
available  for  use  on  all  devices  except  the  T-5,  C-5/C- 141 ,  T-4, 
F-4E/F-4G,  and  F-15  ATDs,  and  the  E-3A  flight  simulator. 
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Utility  and  utilization.  Record/playback  was  among  the  lowest  rated 
of  all  instructional  features.  At  two  sites  (viz.,  T-50,  T-51),  it  was 
fairly  easy  to  use  and  the  ratings  were  in  the  moderate  range.  At  each  of 
the  remaining  sites,  however,  record/playback  was  rated  difficult  and 
time-consuming  to  implement,  and  operational ly  unreliable.  Over  70%  of 
the  B-52  WST,  A-10,  A-10  (EW),  and  E-3A  mission  simulator  Sis  reported 
never  having  used  the  feature. 

Comparisons  across  MAJCQMs  and  levels  of  training.  The  utility  and 
util ization  of  record/playback  did  not  differ  across  the  MA JCOMs .  There 
was  only  one  significant  comparison  across  the  two  levels  of  training: 

E-3A  flight  simulator  RTU  Sis  rated  record/playback  higher  in  potential 
training  value  than  did  E-3A  flight  simulator  CTU  Sis. 

Automated  Performance  Feedback  (Table  20) 

Incidence.  Automated  performance  feedback  was  surveyed  at  each 
non-EW  TAC  ATD  site.  It  was  availaole  for  use  only  on  the  F-4E  ATD  and 
the  E-3A  mission  simulator. 

Utility  and  utilization.  Automated  performance  feedback  was 
apparently  easy  to  use  on  the  E-3A  mission  simulator,  and  its  frequency  of 
use  was  in  the  low  to  moderate  range  of  the  scale.  In  contrast,  the 
feature  was  difficult  and  time-consuming  to  implement  on  the  F-4E  ATD,  and 
its  frequency  of  use  was  at  the  low  end  of  the  scale.  In  fact,  the 
majority  of  F-4E  Sis  reported  never  having  used  the  feature,  preferring 
instead  to  "freeze"  the  mission  and  give  verbal  feedback;  and  over  30% 
were  unaware  of  its  availability. 

Comparisons  across  MAJCQMs  and  levels  of  training.  Comparisons 
across  the  MAJcOMs  were  precluded  since  data  were  collected  only  from  TAC 
ATUs .  On  four  of  the  five  devices  surveyed,  the  RTU  Sis'  mean  rating  of 
the  potential  training  value  was  greater  than  that  of  the  CTU  Sis.  The 
difference  was  significant  only  for  the  E-3A  flight  simulator,  however. 

Hard  Copy  (Table  21) 

Incidence.  Hard  copy  was  surveyed  at  every  ATD  site.  It  was 
available  for  use  on  all  devices  except  the  T-50,  T-51,  C-5/C-141, 
CH-3/HH-53,  T-4,  and  A-10  (EW)  ATUs. 

Utility  and  utilization.  The  frequency  of  use  of  hard  copy  was 
uniformly  low,  with  mean  ratings  all  in  the  "rarely”  to  "occasionally" 
range  of  the  scale.  The  only  successful  implementation  of  this  feature 
appeared  to  be  on  the  T-5  ATD,  where  utility  and  ease  of  use  were  very 
high.  Yet,  it  was  used  only  once  every  two  to  four  missions,  on  the 
average.  Hard  copy  was  one  of  the  most  problematic  features  surveyed. 

For  example,  on  the  C- 130  ATD,  it  was  said  to  be  seldom  operational  and 
time-consuming  to  implement.  On  the  B-52  WST,  it  was  said  to  yield  output 
that  was  difficult  to  interpret.  And  on  the  F-4G  (EW)  ATD,  it  was  called 
"unreliable"  by  28%  of  the  Sis  and  was  either  never  operated  or  presumed 
unavailable  by  53%  of  the  Sis. 


Table  20.  Automated  Performance  Feedback 


Mean  Ratings  (and  Standard  Deviations)  of  the  Frequency  of  Use, 

Ease  of  Use,  Amount  of  Training  Received,  Adequacy  of  Training  Received, 

Training  Value,  and  Potential  Training  Value  by  RTU  (R)  and  CTU  (C)  Simulator  Instructors 
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utilization  of  the  hard  copy  feature  did  not  differ  across  the  MAJCOMs. 
There  was  only  one  significant  comparison  across  the  two  levels  of 
training:  F-15  CTU  Sis  received  more  training  in  the  use  of  hard  copy  than 
did  F-15  RTU  Sis. 

Performance  Scoring  (Table  22} 

Incidence.  Performance  scoring  was  surveyed  at  every  EW  ATD  site. 

The  feature  was  available  for  use  on  the  T-5,  B-52  WST,  and  A-10  devices. 

Utility  and  utilization.  Despite  reports  of  "unreliability"  on  the 
B-52  and  A-10  ATDs,  performance  scoring  generally  received  higher  utility 
and  utilization  ratings  than  the  other  feedback  features  (e.g.,  record/ 
playback  and  hard  copy).  On  the  T-5  ATD,  performance  scoring  was  used 
between  two  and  four  times  each  mission,  on  the  average.  A-10  KTU  Sis 
reported  using  the  feature  even  more  frequently,  between  five  and  seven 
times  each  mission,  on  the  average.  Performance  scoring  was  rated  as 
"moderately"  (B-52  WST)  to  "extremely"  (T-5)  useful. 

Comparisons  across  MAJCOMs  and  levels  of  training.  Comparisons 
across  the  MaJcOMs  were  precluded  since  data  on  this  feature  were 
collected  from  relatively  few  ATDs.  There  was  only  one  significant 
comparison  across  the  two  levels  of  training:  A-10  RTU  Sis  used 
performance  scoring  more  frequently  than  did  A-10  CTU  Sis  (an  average  of 
five  to  seven  times  a  mission  versus  once  every  two  to  four  missions, 
respectively) . 

Amount  and  Adequacy  of  Training  Received  by  Simulator  Instructors 

Sis  generally  received  a  greater  amount  of  training  in  the  use  of 
those  features  that  were  also  rated  higher  in  utility  and  utilization. 

This  was  indicated  by  the  significant  positive  intercorrelations  between 
the  ratings  of  each  AIF  on  the  five  survey  questions.  There  were  clear 
differences  in  the  amount  of  training  received  across  the  MAJCOMs.  TAC 
mean  ratings  (non-electronic  warfare)  were  lower  than  those  of  the  other 
MAJCOMs  for  over  81%  of  the  features  surveyed.  Moreover,  TAC  Sis  tended 
to  characterize  their  training  as  "informal."  In  contrast,  ATC,  MAC,  and 
SAC  Sis  (non-EW)  tended  to  characterize  their  training  as  "formal." 

Observations  concerning  the  adequacy  of  training  received  by  the  Sis 
must  be  considered  tentative,  since  the  relevant  data  were  collected  only 
during  the  EW  phase  of  the  project.  Nevertheless,  the  high  ratings 
suggest  that  the  Sis  from  ATC,  MAC,  SAC,  and  TAC  would  each  characterize 
their  training  as  adequate. 


Some  additional  data  of  interest  were  collected  during  the  EW  phase 
of  the  project.  These  related  to  the  relative  impact  of  "initial"  and 
"refresher"  training  on  AIF  utility  and  utilization.  The  amount  of  formal 
initial  training  appeared  to  relate  positively  to  the  magnitude  of  the 
ratings.  For  example,  T-4  and  T-5  Sis  received  a  greater  amount  of  formal 
training  than  did  other  Sis  (classroom  instruction  accounted  for  54%  and 
28%  of  initial  training,  respectively),  and  the  T-4  and  T-5  ATDs  were 
probably  the  most  favorably  evaluated  devices.  The  impact  of  refresher 


Table  22.  Performance  Scoring 


training  was  less  clear.  For  example,  T-4  and  A-10  ( EW )  Sis  received  a 
greater  amount  of  refresher  training  than  did  other  Sis  (42%  of  T-4  Sis 
and  45%  of  A-10  Sis  received  refresher  training  at  least  once  within  the 
immediately  preceding  year),  but  the  A-10  ratings  tended  to  be  much  lower 
than  those  of  the  T-4. 


IV.  DISCUSSION 
General  Trends 

Interrelations  Among  the  Ratings 

The  results  indicated  that  an  AIF  was  used  to  the  extent  that:  Sis 
were  trained  in  its  use,  it  was  easy  to  use,  and  it  had  apparent  training 
value.  However,  training  value  was  clearly  the  most  significant  predictor 
at  almost  every  ATD  site  surveyed.  What  can  be  concluded  from  these 
facts?  Unfortunately,  correlational  findings  do  not  logically  imply 
causality;  they  merely  reflect  the  presence  of  a  relationship  among 
variables.  In  this  case,  however,  it  seems  reasonable  to  assume  that 
particular  AIFs  were  used  more  frequently  because  they  were  more  useful. 
Indeed,  assuming  that  the  training  value  of  an  AlF  did  not  affect  its  use 
is  clearly  implausible. 

How  can  the  fact  be  explained  that  the  remaining  variables  (viz., 
ease  of  use,  amount/adequacy  of  training  received)  did  not  account  for 
much  of  the  variability  in  frequency  of  use?  This  fact  suggests  that  Sis 
would  not  avoid  using  a  particular  feature,  even  if  it  were  complicated  to 
use,  as  long  as  they  believed  that  it  would  help  accomplish  mission 
objectives. 

Overview  of  the  Rating  Data 

There  were  two  general  trends  revealed  in  the  overview  of  the  rating 
data  (Figures  1  through  6).  The  first  had  to  do  with  differences  in 
utility  and  utilization  among  the  various  types  of  features.  Thus, 
training  management,  variation  of  task  difficulty/fidelity,  and  monitoring 
features  tended  to  receive  the  highest  ratings,  whereas  briefing  and 
feedback  features  tended  to  receive  the  lowest  ratings.  The  fact  that 
this  trend  was  observed  at  every  ATD  site  suggests  that,  regardless  of 
differences  in  particular  mission  requirements.  Sis  adopted  a  common 
strategy  for  the  use  of  instructional  features.  Apparently  the  strategy 
was  one  in  which  the  use  of  AIFs  was  concentrated  during  the  actual 
mission  performance  (e.g.,  for  training  management).  Although  certain 
AIFs  could  be  used  outside  the  mission  context  (e.g.,  for  briefing),  they 
were  used  less  often.  There  were  two  notable  exceptions  to  this  trend. 
Automated  adaptive  training,  a  training  management  feature,  received 
extremely  low  utility  and  utilization  ratings,  whereas  performance 
scoring,  a  feedback  feature,  generally  received  high  ratings.  These  two 
AIFs  will  oe  discussed  in  greater  detail  below. 

The  second  general  trend  revealed  in  the  overview  of  the  data  had  to 
do  with  differences  in  the  ratings  of  certain  features  across  the  two 


i 


levels  of  training.  The  differences  between  the  RTU  and  CTU  Sis'  use  of 
AIFs  appeared  to  reflect  a  critical  distinction  between  the  two  types  of 
missions.  RTU  missions  tended  to  stress  the  acquisition  of  discrete 
skills  and  procedures,  whereas  CTU  missions  tended  to  maintain  the  broader 
mission  context.  Thus,  RTU  Sis  were  more  likely  to  interrupt  or  otherwise 
alter  the  mission  in  order  to  afford  students  sufficient  opportunity  for 
practice  and  feedback;  they  more  frequently  invoked  features  such  as 
freeze,  reset,  environmental,  and  parameters  and  procedures  monitoring  in 
order  to  accomplish  this  objective.  In  contrast,  CTU  Sis  spent  less  time 
interrupting  the  mission  in  order  to  achieve  a  more  continuous  and 
realistic  scenario.  Programmed  scenarios  were  ideally  suited  for  this 
purpose,  and  they  were  more  frequently  invoked  by  CTU  Sis. 

It  is  important  to  point  out,  however,  that  the  differences  described 
above  were  not  evidenced  at  each  ATD  site.  In  fact,  there  were  no 
significant  differences  between  RTU  and  CTU  ratings  on  over  40%  of  the 
devices,  excluding  ATC  (viz.,  CH-3/HH-53,  T-4,  B-52  WST,  F-4E/F-4G,  F-4G 
EW,  E-3A  mission  simulator).  Even  on  devices  where  differences  were 
observed  (e.g.,  F - 1 5  ATD),  the  majority  of  features  were  used  similarly  by 
RTU  and  CTU  Sis.  Thus,  Sis  were  more  similar  in  their  use  of  AIFs  than 
they  were  different,  both  groups  used  the  same  features  to  one  degree  or 
another,  and  each  ATD  could  readily  accommodate  either  type  of  training. 

Utility  and  Utilization  Ratings 


Briefing  Features 

The  briefing  features,  recorded  briefing  and  uemonstration,  tended  to 
be  among  the  least  valued  and  least  used  features  of  those  surveyed.  The 
Sis'  comments  suggested  that  there  were  two  major  reasons  for  the  low 
ratings.  First,  Sis  generally  preferred  to  brief  students  themselves. 

This  was  probably  a  more  reasonable  strategy  since  individual  students' 
needs  might  differ,  and  a  "generic"  briefing  might  not  be  appropriate  for 
all  students.  Second,  the  low  ratings  undoubtedly  reflected  operational 
difficulties  and  limitations  associated,  particularly,  with 
demonstration.  The  most  commonly  reported  problems  were  "time-consuming 
implementation"  and  "unreliability."  This  would  seem  unfortunate,  since, 
according  to  social  learning  theory,  "a  large  amount  of  human  learning  is 
done  vicariously,  through  observing  another  person  making  the  skilled 
responses...  and  then  trying  to  imitate  the  response  of  the  model" 

(Hilgard  &  Bower,  1975,  pp.  599-605).  Nevertheless,  empirical  evidence 
from  the  aircrew  flight  training  literature  suggests  that  the  use  of 
demonstration  is  no  more  effective  for  training  than  is  simple  performance 
feedback  (Hughes  et  al.,  1979). 

The  potential  training  value  mean  ratings  were  unremarkable.  They 
were  similar  across  the  ATDs  and  were  all  in  the  moderate  range  of  the 
scale.  None  of  the  devices  surveyed  had  an  instructor  tutorial 
capability,  and  it  is  therefore  difficult  to  discuss  the  utility  and 
utilization  of  this  feature.  However,  the  few  Sis  that  commented  on 
instructor  tutorial  apparently  preferred  "hands-on"  experience  and/or 
"face-to-face"  tutorials  for  learning  the  operation  of  the  instructor 
console. 
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Training  Management  Features 


Freeze  and  reset  were  among  the  highest  rated  of  all  features 
surveyed.  This  was  true  for  virtually  every  device  that  was  surveyed. 

The  ability  to  temporarily  suspend  a  mission  in  order  to  instruct  a 
student,  and  then  allow  the  mission  to  continue,  appeared  to  be  essential 
for  effective  training  management,  especially  for  RTU  training,  in  which 
the  acquisition  of  discrete  skills  and  procedures  was  stressed.  Yet,  the 
empirical  evidence  for  the  training  effectiveness  of  these  features  is 
less  clear.  Bailey,  Hughes,  and  Jones  (1980)  successfully  used  freeze  and 
reset  in  their  application  of  the  behavioral  "backward  chaining"  paradigm 
to  air-to-surface  weapon  delivery  training.  On  the  other  hand,  Hughes  et 
al.  (1982)  were  unable  to  significantly  improve  carrier  glideslope 
tracking  performance  by  using  freeze  as  opposed  to  a  more  conventional  "no 
freeze"  training  approach.  Their  approach  was  to  freeze  the  simulator 
whenever  a  glideslope  error  was  detected.  The  simulator  was  then  returned 
to  the  previous  position,  with  appropriately  configured  angle-of-attack 
and  airspeed,  and  the  student  was  allowed  to  try  again. 

Crash-kill  override  was  typically  "left  on"  for  most  training 
applications  in  order  to  avoid  reinitializing  the  simulator  following  a 
"crash"  or  "kill"  and,  thus,  preserve  instruction  time.  "Crashes"  were 
allowed  to  occur  more  frequently  on  ATC  trainers  (viz.,  T-50,  T-51). 
Several  ATC  Sis  commented  that  entry-level  pilots  must  be  taught  to  be 
sensitive  to  dangerous  conditions.  Experiencing  a  simulated  crash  is 
apparently  one  way  to  accomplish  this  objective. 

Automated  adaptive  training  received  the  lowest  utility  and 
utilization  ratings  of  the  training  management  features;  however,  it  is 
difficult  to  draw  any  firm  conclusions  since  the  feature  was  operational 
on  only  one  ATD,  the  F-4E/F-4G.  Notwithstanding  the  low  ratings,  there  is 
considerable  evidence  that  automated  adaptive  training  has  training 
utility  (Brown,  Waag,  &  Eddowes,  1975;  Charles,  Johnson,  &  Swink,  1971, 
1973;  Charles,  Willard,  &  Healey,  1975;  Feuge,  Charles,  &  Miller,  1973). 
According  to  Brown  et  al.  (1975),  the  major  difficulties  with  automated 
adaptive  training  are  limited  training  scenarios,  the  high  cost  of 
software  implementation,  the  lack  of  formal  instructor  training  in  the  use 
of  the  feature,  and  the  lack  of  training  directives  to  implement  such 
training. 

Programmed  mission  scenarios  were  used  to  "streamline"  a  training 
session  by  freeing  a  simulator  instructor  to  perform  other  important 
duties,  such  as  monitoring  student  performance  and  giving  feedback. 
Programmed  scenarios  were  especially  useful  for  CTU  missions,  which  tended 
to  be  interrupted  less  often  than  were  RTU  missions.  However,  as 
discussed  earlier,  there  were  certain  operational  limitations  associated 
with  the  use  of  programmed  mission  scenarios.  Thus,  the  programming  of 
scenarios  was  tedious  and  difficult,  and  there  were  few  available 
scenarios  at  most  training  sites.  Moreover,  those  that  were  available 
were  frequently  characterized  as  "unreliable,"  "limited,"  or  "outdated." 
Although  manual  mission  control  afforded  the  SI  a  greater  degree  of 
instructional  flexibility,  it  required  considerable  time  and  effort. 


Variation  of  Task  Difficulty/Fidelity  Features 

Most  emergencies  can  be  safely  dealt  with  only  under  simulated 
conditions,  and  the  frequent  insertion  of  simulated  malfunctions  was 
characteristic  of  every  training  mission  that  was  observed.  However,  the 
same  problems  that  limited  tne  use  of  programmed  mission  scenarios  also 
limited  the  use  of  automated  malfunction  insertion  at  the  majority  of  ATD 
sites.  It  was  said  that  it  was  time-consuming  to  program  and  implement 
and  unreliable,  and  did  not  always  reflect  mission  profiles.  Roughly  one 
in  five  Sis  preferred  to  insert  malfunctions  manually,  despite  the  greater 
effort  that  was  required. 

Of  the  two  "fidelity"  features  that' were  surveyed,  environmental  and 
motion,  environmental  appeared  to  be  the  least  problematic.  The  utility 
and  utilization  ratings  of  the  environmental  feature  were  uniformly 
favorable,  except  for  the  E-3A  mission  simulator,  which  had  a  very  limited 
environmental  capability.  Platform  motion  cueing  was  more  frequently  used 
and  was  considered  a  more  useful  training  feature  than  was  environmental; 
however,  at  every  ATD  site,  the  motion  system  was  difficult  to  maintain 
and  was  sometimes  inoperable. 

As  described  in  the  Results  section,  motion  received  higher  utility 
ratings  from  non-fighter  Sis  (e.g.,  C - 130,  E-3A  flight  simulator)  than 
from  fighter  Sis  (e.g.,  F-15).  This  difference  was  probably  due  to  the 
fact  that  platform  motion  cueing  systems  are  not  capable  of  high-fidelity 
simulation  of  fighter  aircraft  movement.  Empirical  evidence  suggests  that 
platform  motion  cueing  is  more  useful  in  non-fighter  applications.  Thus, 
Ricard,  Parrish,  Ashworth,  and  Wells  (1981)  found  that  platform  motion 
cueing  was  effective  for  helicopter  hover  simulation  training,  whereas 
Martin  and  Waag  (1978a,  1978b)  found  that  it  was  ineffective  for  basic 
contact  maneuvers  and  aerobatics.  The  general  utility  of  motion  was  also 
questioned  by  Cyrus  (1978),  who  concluded  in  his  literature  review  that, 
for  most  tasks,  the  elimination  of  platform  motion  cues  does  not  reduce 
training  effectiveness. 

The  final  feature  in  this  group,  partial  freeze,  was  most  often  used 
as  a  substitute  for  total  system  freeze;  and  both  features  shared  high 
utility  and  utilization  ratings  for  similar  reasons.  However,  partial 
freeze  also  permitted  Sis  to  vary  the  student's  task  load  by  selectively 
freezing  particular  aircraft  parameters.  Partial  freeze  may  therefore 
offer  certain  advantages  over  total  system  freeze,  since  it  can  be  used 
both  to  manage  training  and  to  vary  task  difficulty. 


Monitoring  Features 

Parameters  monitoring  and  procedures  monitoring  permitted  an  SI  to 
monitor  student  performance  during  a  simulated  mission  by  means  of 
alphanumeric  and/or  graphic  CRT  displays  of  performance  data.  Not 
surprisingly,  parameters  and  procedures  monitoring  were  among  the  highest 
rated  features. 


The  monitoring  displays  were  usually  viewed  at  a  remote  instructor's 
console.  Alternate  but  less-sophisticated  capabilities  for  monitoring 
student  performance  from  a  remote  console  included  repeaters  (replicas  of 
flight  instruments)  and  annunciators  (indicators  that  are  directly  linked 
to  aircraft  controls  and  switches).  On  some  devices  (e.g.,  C-130  ATD, 
CH-3/HH-53  ATD),  the  instructor  console  was  located  within  the  simulator 
cockpit.  In  these  situations,  Sis  could  monitor  performance  by  means  of 
displays  or  by  directly  viewing  the  student  "over-the-shoulder." 
Interestingly,  repeaters,  annunciators,  or  "over-the-shoulder"  were  often 
the  preferred  means  of  monitoring  student  performance.  This  suggests  that 
Sis,  most  of  whom  are  experienced  aircrew  members  themselves,  find  it 
easier  to  monitor  parameters  and  procedures  in  ways  similar  to  those  used 
during  actual  flying.  This  also  suggests  that  the  utility  of  remote 
parameters  and  procedures  monitoring  displays  will  depend  on  the  format  in 
which  performance  information  is  presented;  e.g.,  digital  displays  of 
round  dial  instrument  readings  will  be  unacceptable  to  most  Sis. 

Feedback  Features 

Record/playback  and  hard  copy  were  available  for  use  on  the  majority 
of  ATDs  surveyed.  The  ratings  of  these  features  were  uniformly  low,  with 
the  exception  of  the  ATC  devices  (e.g.,  record/playback  on  the  T-50  and 
T-51  ATDs,  and  hard  copy  on  the  T-5  ATD).  The  utility  of  the  record/ 
playback  and  hard  copy  features  was  limited  by  operational  problems  that 
discouraged  their  use.  Both  features  were  difficult  and  time-consuming  to 
implement  and  were  operationally  unreliable.  Had  these  features 
functioned  reliably  and  efficiently,  they  might  have  been  used  more 
often.  Indeed,  the  giving  of  feedback  was  a  normal  instructional 
procedure  at  every  ATD  site,  although  such  feedback  was  more  often  given 
verbally  than  by  means  of  AIFs. 

The  amount  of  data  collected  on  automated  performance  feedback  and 
performance  scoring  was  too  small  to  support  any  firm  conclusions. 
Automated  performance  feedback  was  operational  on  only  two  devices,  the 
F-4E  ATD  and  the  E-3A  mission  simulator.  The  majority  of  F-4E  Sis  had 
never  used  the  feature  because  it  was  difficult  and  time-consuming  to 
implement.  The  E-3A  mission  simulator  Sis  used  it  infrequently.  The 
potential  training  value  ratings  suggested  that  automated  performance 
feedback  could  be  useful,  especially  for  KTU  training.  One  advantage  of 
automated  performance  feedback  is  that  it  is  relatively  unobtrusive,  since 
it  merely  presents  an  "error  cue"  while  allowing  the  mission  to  continue. 
By  using  automated  performance  feedback  during  RTU  missions,  where  errors 
are  more  frequent,  the  instruction  time  that  is  normally  used  for  verbal 
feedback  could  be  saved. 

In  contrast  to  the  other  feedback  features,  performance  scoring 
received  generally  favorable  utility  and  utilization  ratings.  There  are 
two  possible  reasons  for  the  high  ratings.  First,  the  feature  was  easy  to 
use  and  was  reliable.  Second,  as  with  automated  performance  feedback, 
performance  scoring  saved  instruction  time  by  "summarizing"  performance 
automatically. 


The  only  experimental  evidence  of  the  utility  of  the  feedback 
features  concerns  record/playback.  Hughes,  Hannon,  and  Jones  (1979)  found 
that  the  periodic  use  of  a  replay  of  student  performance  was  more 
effective  in  reducing  errors  during  subsequent  performance  than  was  the 
use  of  an  instructor-recorded  "demonstration."  However,  record/playback 
was  no  more  effective  than  simple  practice. 

Training  Received  by  Simulator  Instructors 

As  described  in  a  previous  section  of  this  report,  there  were 
considerable  differences  across  ATD  sites  in  the  amount  and  type  of 
training  received  by  the  Sis.  TAC  Sis  apparently  received  less  training 
than  did  ATC,  MAC,  or  SAC  Sis;  and  TAC  training  was  more  often 
characterized  as  "informal."  Vet  the  electronic  warfare  results  indicated 
that  TAC,  ATC,  MAC,  and  SAC  Sis  each  rated  their  training  as  adequate, 
despite  the  stated  differences  in  amount  and  type.  This  suggests  that 
there  may  not  be  a  "best"  way  to  train  Sis,  but  an  important  question 
needs  to  be  answered  before  any  firm  conclusions  can  be  drawn:  What, 
precisely,  are  the  appropriate  criteria  for  "adequate"  training?  These 
will  need  to  be  empirically  determined. 


V.  RECOMMENDATIONS 

Improve  the  Training  of  Sis 

It  is  clear  from  this  survey  that  the  existing  AIF  capabilities  of 
Air  Force  ATDs  have  not  been  fully  explored.  This  is  partly  due  to 
operational  problems  that  h.'ve  precluded  the  use  of  certain  features,  but 
it  is  also  likely  due  to  insufficient  training  of  Sis. 

The  extensive  AIF  capabilities  of  modern  ATDs  provide  an  inherently 
flexible  and  dynamic  training  environment.  Many  different  AIFs  can  be 
implemented  singly  or  in  combination  at  various  points  during  a  simulated 
mission.  What  should  Sis  be  taught  about  implementing  AIFs?  Whatever 
form  the  training  of  Sis  takes  (the  results  of  the  survey  suggest  that 
both  formal  and  informal  training  may  be  effective).  Sis  must  be  taught 
not  only  how  to  use  the  available  AIFs  but  also  how  to  use  them 
effectively  (i.e.,  in  ways  that  will  maximize  the  acquisition  and 
retention  of  aircrew  skills). 

Specify  Guidelines  for  Using  AIFs  Effectively 

Guidelines  for  using  AIFs  effectively  need  to  be  expressed  in 
operational  terms.  It  is  not  sufficient  for  Sis  to  know  how  to  use  a 
feature;  they  must  also  know  when  to  use  it.  That  is,  Sis  must  know  when 
to  use  a  feature  during  a  mission  in  order  to  maximize  student  performance. 

Guidelines  for  effective  AIF  use  still  need  to  be  specified, 
however.  Such  guidelines  cannot  be  derived  from  surveys.  They  require 
the  conducting  of  experiments  that  compare  the  effects  of  implementing  or 
not  implementing  a  particular  AIF  on  various  performance  criteria  (e.g., 
deviation  from  glideslope  or  accuracy  of  weapon  delivery).  The  few 
experiments  of  this  type  that  have  been  conducted  were  mentioned  in 
previous  sections  of  this  report.  Many  more  are  needed. 


Improve  the  Operability  of  Advanced  Instructional  Features  (AIFs) 

If  the  instructional  capability  of  ATDs  is  to  be  fully  realized,  AIFs 
will  need  to  be  made  more  reliable  and  user-friendly.  This  survey 
revealed  that  there  were  operational  problems  with  one  or  more  features  at 
every  ATD  site.  A  feature  cannot  be  used  effectively  if  it  is  difficult, 
time-consuming,  or  simply  impossible  to  implement.  ATD  design 
requirements  must  ensure  that  the  full  range  of  instructional  capabilities 
can  be  utilized  and  maintained. 
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Table  A-1 .  Simulator  Instructors  (Sis)  Surveyed  in  Phase  I 
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George 
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George 
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Luke 
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Tinker 
AFB,  OK 
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Replacement 


Continuation 


Replacement 
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Replacement 
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Replacement 
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Replacement 
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Table  A-2. 

Simulator 

Instructors  (Sis) 

Surveyed 

in  Phase 

II 

Command  ATD 

ATU-Sites 

Level  of 
training 

Type 
of  SI 

N 

Instructor 

hours 

ATC  T-50 

Williams 
AFB,  AZ 

Basic 

IP 

29 

173.6 

(169.5) 

T-51 

Williams 
AFB,  AZ 

Basic 

IP 

21 

129.4 

(96.8) 

MAC  C-5A 

Altus 

AFB,  OK 

Formal 

IP, IFE 

29 

511.1 

(452.1) 

Dover 

AFB,  DE 

Operational 

IP, IFE 

17 

454.1 

(386.8) 

C-141 

Altus 

AFB,  OK 

Formal 

IP, IFE 

40 

582.5 

(531.4) 

McGuire 

AFB,  OK 

Operational 

IP, IFE 

13 

1174.6 

(1504.1) 

C-130 

Little  Rock 
AFB,  AR 

Formal 

IP, IFE 

21 

126.8 

(80.4) 

Little  Rock 
AFB,  AR 

Operational 

IP, IFE 

13 

419.8 

(189.9) 

CH-3 

Kirtland 
AFB,  NM 

Formal 

IP, IFE 

6 

169.2 

(94.4) 

Kirtland 
AFB,  NM 

Operational 

IP, IFE 

5 

263.4 

(132.7) 

HH-53 

Kirtland 
AFB,  NM 

Formal 

IP, IFE 

5 

139.2 

(149.8) 

Kirtland 
AFB,  NM 

Operational 

IP, IFE 

12 

482.5 

(452.1) 

SAC  FB-111A 

PI attsburgh 

AFB,  NY 

Transition 

IP, IKN 

17 

797.1 

(693.3) 

Plattsburgh 

AFB,  NY 
Pease  AFB,  NH 


Operational  IP, I R  N  45 


353.0 

(818.4) 


Table  A-3. 

Simulator  Instructors  (Sis) 

Surveyed  in 

Phase 

III 

Command  ATD 

ATD-Sites 

Level  of 
training 

Type 
of  SI 

N 

Instructor 

hours 

ATC  T-5 

Mather 

AFB,  CA 

Basic 

IEW 

19 

287.4 

(276.0) 

SAC  T-4  (B-52) 

Castle 

AFB,  CA 

Transition 

IEW 

20 

731.2 

(754.6) 

Mather 

AFB,  CA 

Operational 

IEW 

8 

188.9 

(224.6) 

WST  (B-52) 

Castle 

AFB,  CA 

Transition 

IEW, I AG 

9 

674.9 

(747.8) 

Wurtsmith 

AFB,  MI 

Operational 

IEW, I AG 

6 

396.2 

(231.9) 

*FB-1 1 1A 

PI  attsburgh 
AFB,  NY 

Transition 

IRN 

11 

677.3 

(426.2) 

PI  attsburgh 
AFB,  NY 

Operational 

IRN 

9 

175.6 

(213.8) 

Pease 

AFB,  NH 

Operational 

IRN 

12 

210.2 

(182.7) 

TAC  F-4G 

George 

AFB,  CA 

Replacement 

IEW 

13 

128.4 

(116.1) 

George 

AFB,  CA 

Continuation 

IEW, IP 

19 

73.1 

(56.4) 

A-10 

Davis-Honthan 
AFB,  AZ 

Replacement 

IP 

16 

98.8 

(85.8) 

England 

AFB,  LA 

Continuation 

IP 

17 

52.6 

(37.4) 

T5S 


*  Data  from  these  sites  were  collected  during  Phase  II. 
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APPENDIX  B:  SURVEYS  FOR  PHASES  I,  II,  AND  III 


Tyiv 
■  ‘V* 


PHASE  I  QUESTIONNAIRE 

ADVANCED  INSTRUCTIONAL  FEATURES  -  IP  SURVEY 

Name _ Rank _ Squadron _ Date _ 

FLYING  EXPERIENCE: 

Aircraft  Total  Hours  IP  Hours 


SIMULATOR  EXPERIENCE: 
Simulator 


Total  Hours 


IP  Hours 


BRIEFLY  DESCRIBE  A  "TYPICAL"  TRAINING  SESSION  ON  THIS  SIMULATOR 


PHASE  I  QUESTIONNAIRE  (continued) 


Please  familiarize  yourself  with  these  instructional  features  and  their 
definitions: 

Instructor  Pilot  Tutorial  -  provides  the  IP  with  self-paced  programmed 
instruction  in  the  capabilities  aid  use  of  the  flight  simulator. 

Reset  -  permits  instructor  to  "return"  the  simulated  aircraft  to  a  stored 
set  of  conditions  and  parameters. 

Total  System  Freeze  -  permits  instructor  to  interrupt  and  suspend 
simulated  flicjit  by  freezing  all  system  parameters. 

Recorded  Briefing  -  permits  instructor  to  provide  student  with  information 
about  a  structured  training  session  through  audio/visual  media 
presentation . 

Demonstration  -  permits  instructor  to  demonstrate  aircraft  maneuver (s)  by 
prerecording  and  subsequently  playing  back  a  standardized  segment  of 
simulated  flight. 

Record/Playback  -  permits  instructor  to  record  and  subsequently  playback 
all  events  that  occurred  during  a  segment  of  simulated  flight. 

Environmental  -  permits  instructor  to  vary  environmental  conditions  such 
as  wind  direction  and  velocity,  turbulence,  temperature,  visibility,  etc. 

Automated  Malfunction  Insertion  -  permits  instructor  to  pre-program  a 
sequence  of  aircraft  component  malfunctions  and/or  emergency  conditions. 

Flight  System  Freeze  -  permits  instructor  to  simultaneously  freeze  flight 
control  and  propulsion  systems,  latitude,  longitude,  altitude,  and  heading. 

Position  Freeze  -  permits  instructor  to  simultaneously  freeze  latitude  and 
longitude. 

Parameter  Freeze  -  permits  instructor  to  freeze  any  one  or  combination  of 
flight  parameters. 

Crash  and/or  Kill  Override  -  permits  instructor  to  allow  simulated  flight 
to  continue  without  interruption  following  a  "crash"  or  "kill." 

Motion  -  permits  instructor  to  vary  platform  motion  system  cues  such  as 
roll,  pitch,  lateral,  vertical,  etc. 

Automated  Per formance  Feedback  -  provides  student  with  visual  and/or 
'  auditory  signals  (including  verbal  messages)  that  identify  performance 

[  deficiencies. 

Hard  Copy  -  provides  a  record  of  alphanumeric  and/or  graphic  performance 
data  rrom  the  automated  performance  measurement  system  for  debriefing 
purposes . 

\  Automated  Adaptive  Training  -  computer -con trolled  variations  in  task 

\  difficulty,  complexity,  and  sequence  based  on  pilot's  performance. 

Programmed  Mission  Scenarios  -  computer-controlled  standardized  training 
sessions  based  on  pre-programmed  event  sequences. 


PHASE  I  QUESTIONNAIRE  (Continued) 

J.  Hm  «ith  training  did  m  ra coin*  In  th*  in*  of  aach  tnitmCt  to«a'  faatur*'  (Chtclt  the  awreprtata  spac*.  Plim 
co***nt  ii  to  vhcthtr  tn*  traln'id  in  fomal  or  Infonoat.) 


Inttnactar  Rllot  Tutorial 


unarallabl*/  non*  /alniaal/  to**  /aiodtrata/contldartbla/grnat/jrtatttt/ 

/ _ /  / _ / _ /  /  /  / 


/  I  t _ /  /  /  /  /  / 


Total  Syataa  Franz* 


/  /  /  / 


Racordnd  trl*fl«f 


/  /  /  / 


Donontt  ratio* 


Record /Playback 


Cnal  iMMt  al 


Autoaatad  Nal function  Intartlo* 


/ _ /  _ / 


Fllft  Sytto*  Frooto 


/  /  / 

- TT - 1 - 


Rot  It  Ion  Franz* 


llrpator  Franco 


Craan  and/or  RI11  Oaarridt 


Auto*  at  ad  Rarfaraanc*  Faadback 


Hard  Co*y 


•utoMt ad -Motion  Training 


Ri  oya— d  Wat  Ian  Scenario* 


,’**«*.*  *  i  1 


*i*  .*  ,<  \»  »**,i!‘,i 


V.V 


.  f  ..  >'  *  .  .  *  V  '  r  ’•  .  »  ■:  I.  '.  V,f  2.t  l>i  •■}'»  a'  <■  ’■'«.• 

*.  •»♦  .  ,  ..i> 4 •,'  £*.  \  v  •*  ..  4  •  i  >  ^  -  It.  •  **  '  ■ 


*!♦-*<*  * 


‘v*ei 
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PHASE  II  QUESTIONNAIRE 

ADVANCED  INSTRUCTIONAL  FEATURES  -  IP  SURVEY 

Name _ R  ank _ Sq  u  ad  ron _ Da  te _ 

FLYING  EXPERIENCE: 

Aircraft  Total  Hours  IP  Hours 


SIMULATOR  EXPERIENCE: 

Simulator  Total  Hours 


IP  Hours 


BRIEFLY  DESCRIBE  A  "TYPICAL"  TRAINING  SESSION  ON  THIS  SIMULATOR: 


! 


GENERAL  COWCNTS  AND/OR  RECOMMENDATIONS 


PHASE  II  Questionnaire  (Continued) 


Please  familiarize  yourself  with  these  instructional  features  and  their 
definitions:  F-or  eacn  feature,  insert  1  (available)  or  0  unavailable^: 

nm 


Instructor  Pilot  Tutorial  -  provides  the  IP  with  self-paced  programned 
instruction  in  the  capabilities  and  use  of  the  flight  simulator. 

Reset  -  permits  instructor  to  "return"  the  simulated  aircraft  to  a 
stored  set  of  conditions  and  parameters. 

Total  System  Freeze  -  permits  instructor  to  interrupt  and  suspend 
simulated  flight  by  freezing  all  system  parameters. 

Recorded  Briefing  -  permits  instructor  to  provide  student  with 
information  about  a  structured  training  session  through  audio/visual 
media  presentation. 

Demonstration  -  permits  instructor  to  demonstrate  aircraft  maneuver^) 
by  prerecording  and  subsequently  playing  back  a  standardized  segment 
of  simulated  flight. 

Record/Playback  -  permits  instructor  to  record  and  subsequently 
playback  all  events  that  occurred  during  a  segment  of  simulated  flight. 

Environmental  -  permits  instructor  to  vary  environmental  conditions 
such  as  wind  direction  and  velocity,  turbulence,  temperature, 
visibility,  etc. 

Automated  Malfunction  Insertion  -  permits  instructor  to  pre-program  a 
sequence  of  aircraft  component  malfunctions  and/or  emergency 
conditions. 

Partial  Freeze  -  permits  instructor  to  freeze  various  flight 
parameters  or  parameter  combinations  such  as  altitude,  heading, 
position,  attitude,  flight  system,  etc. 

Crash  and/or  Kill  Override  -  permits  instructor  to  allow  simulated 
flight  to  continue  without  interruption  following  a  "crash"  or  "kill." 

Motion  -  permits  instructor  to  vary  platform  motion  system  cues  such 
as  roll ,  pitch,  lateral,  vertical,  etc. 

Hard  Copy  -  provides  a  record  of  alphanumeric  and/or  graphic 
performance  data  from  the  automated  performance  measurement  system  for 
debriefing  purposes. 

Automated  Adaptive  Training  -  computer-controlled  variations  in  task 
difficulty,  complexity,  and  sequence  based  on  pilot's  performance. 

Programned  Mission  Scenarios  -  computer-controlled  standardized 
training  sessions  based  on  pre-programned  event  sequences. 

Procedures  Monitoring  -  permits  instructor  to  monitor  discrete  actions 
performed  by  the  student  in  accordance  with  a  procedural ly  defined 
checklist. 

Parameters  Monitoring  -  permits  instructor  to  monitor  various 
instrument  readings,  control  settings,  aircraft  states,  or 
navigational  profiles. 
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PHASE  II  QUESTIONNAIRE  (Continued) 

1.  Ho*  often  have  you  used  etc*  instruction*)  future?  (Check  the  appropriate  space.) 


feature 

Instructor  Pilot  Tutorial 
Co— nts: 


/moderately/  /  very  / 

/never /rere ly/occes lone  1  )y/  often  /frequent  ly/frequent ly/most  often 

V-H - r  1  i — L  r  /  ;  ' 


Reset 


V4 — ^ — I — . — t— S — 5- 


Ca— nts: 


Tot* )  System  Freeze 

Coowents: 


l~r - i-  ■*-!  ■  L 


t - ' — 5 - L — r 


Recorded  Briefing 


Demons  tret  Ion 


Record/P  I *)« act 


environmental 


Co— nts: 


Comments: 


Comments: 


Comments: 


L-r-L~r-L - 3 — L 


T - L - 5 - L - 5 - L - 7 - ' 


1  - r~-i — <  L  r '  -^--o  L  i  ‘ 


L-l-LT-L—r  L-T-J-T-L- T-  /  7  / 


*T^»  1 


T - 1 — I - L — 5 - L — l - L 


Autoauted  Malfunction  Insertion 
Comments: 


Partial  Freeze 


Comments: 


J _ _ _ L 


'  \~l  l  ' - 5 - ? - 5 — ‘ — 5 — - r 


Crash  and/or  Mil  Orcrrldt 
Commtnts: 


'  i 1 . 1 


T  <— .  <  i  '  l  '  ,  ' 


Motion 


'  i 1  i  ' - r-  <  '  i  <— i  r  ' 


Hard  Copy 


Comments: 


Co— nts: 


7  rS  - / - -i — r~  1 — r  *-  7  7 


Rut— ted  Adaptive  Training 
Coammts: 


1  r^~7  " / - J - z — 1 - L — 5 - L — 5 - L — r 


Programmed  Mission  Scenarios 
Co— nts: 


1  V1  l  1 - J - L - 1 - L - 5 - L - 5 - L - 7 - ' 


Procedures  itonltortng 

Co— nts: 


L  r~^~?  J — 3 — L — 1 — L — 5 — L — 5 — L — 7 — ' 


Per— tees  Monitoring 


'  1  ‘  1  ' - J - L - l - L - 5 - L - 1 - L - 7 - ' 


Co— nts: 
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PHASE  II  QUESTIONNAIRE  (Continued) 

2.  *»«  eas/  'i  It  to  use  eacn  instruct ional  feature?  IChect  the  appropriate  space. i 


Instructor  ^4 lot  Tutorial 


Total  System  f reete 


Recorded  Briefing 


Demonstration 


Record/P  I  ayti act 


Environmental 


C  i/ments : 


Automated  Malfunction  Insertion 


Partial  freeae 


Crasn  and/or  ail!  Override 


nerd  Cop/ 


Automated  Adaptive  Training 


Programed  Mitt  Ion  Scenario! 


Procedure!  taring 


Parmetert  >tonl taring 

C  ament  t: 


/  most  /  ver/  / 

/  ?  /dtffleult/difflcult/dlfflcult/aoderate/  eas/  /ver/  easy/easiest/ 


i  ‘t  - 
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PHASE  II  QUESTIONNAIRE  (Continued) 

3.  Moo  ouch  training  414  jou  receive  in  tne  use  at  eecn  Instructional  feature?  , Cnees  me  appropriate  space. 
Please  coeaent  as  to  onetner  me  training  oas  foreal  or  informal. i 


Feature 

Instructor  Pilot  Tutorial 
Codnts: 


/  none  /aimaa!/  soae  /aooerate/consloeraole/great/greatest/ 


Reset 


Caaaents: 


Total  S/staa  Freeae 


Recorded  If leftng 


Oeaonstrattoa 


Record/PlayOact 


Environmental 


Com  ants: 


Coaaents: 


plants; 


Clients: 


Ru toasted  Malfunction  Insertion 
dents: 


Partial  Freeie 


Cdents: 


Crasn  and/or  Kill  Override 
(dents: 


Motion 


Hard  Copy 


dents: 


Cdents: 


Rutoaated  Adaptive  Training 
dents; 

Prograaaad  Mission  Scenarios 
dents: 

Procedures  Monitoring 

its: 


Par  teeters  Monitoring 

dents: 
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PHASE  II  QUESTIONNAIRE  (Continued) 

4.  Sate  tne  training  value  of  each  Instructional  feature.  (Check  the  aoproprtate  space.) 


feature 

Instructor  Pilot  Tutorial 

Coalmen  tic 


Reset 


Cements: 


Total  System  Freete 

Cements: 


Recorded  Briefing 

Comment!: 


Demonstration 

Comments: 


Record/P  I a> tack 

Comments: 


environmental 

Comments: 

Automated  Malfunction  Insertion 
Comments: 


Partial  freete 

Comments: 

Crash  and/or  kill  Override 
Comments: 


Motion 


Coaments: 


Hard  Copy 

Coements: 

Automated  Adaptive  Training 
Comments: 

Programmed  Mission  Scenarios 
Comments: 

Procedures  Monitoring 

Comments: 

Parameters  Wnltorlng 

Comments: 


/  ?  /  none  /minimal/  some  /moderate/considerab le/great/greatest/ 
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PHASE  III  QUESTIONNAIRE 


ADVANCED  INSTRUCT  ICNAL  FEATURES  -  EWI  SURVEY 


Name _ Rank _ Squadron _ Date 


FLYING  EXPERIENCE: 

Aircraft  Total  Hours  Instructor  Hours 


SLMJLATCR  EXPERIENCE: 
S  imulator 


Total  Hours 


Instructor  Hours 


1.  What  percent  of  your  initial  instruction  on  simulation  training  consisted  of  formal 
classroom  instruction  and  what  percent  consisted  of  informal  instruction? 

I  %  formal  classroom  %  informal 

»  '  - 

2.  Have  you  had  refresher  training  on  simulation  operation?  yes  no 

(If  no,  skip  next  two  items.) 

\ 

|  a.  How  long  has  it  been  since  you  last  had  refresher  training?  _  weeks 

i  b.  what  percent  of  your  refresher  training  was  formal  and  what  percent  informal? 

i 

_ l  formal  classroom  _ %  informal 


BRIEFLY  DESCRIBE  A  "TYPICAL"  TRAINING  SESSION  ON  THIS  SIMULATOR: 


GENERAL  COMMENTS  AfO/OR  RECOM-eCAIICNS 


PHASE  HI  QUESTIONNAIRE  (Continued) 


Read  the  definitions  of  each  instructional  feature  carefully.  In  the 
space  next  to  each  feature,  write  the  single  number  corresponding  to  the 
statement  that  oest  describes  the  operational  status  of  that  feature: 

0.  The  simulator  has  no  such  capability. 

1.  Capability  present  but  I  have  never  seen  it  operate. 

2.  Capability  present  but  unreliable. 

3.  Capability  present  and  reliable. 


Instructor  Tutorial  -  provides  the  instructor  with  self-paced 
programed  instruction  in  the  capabilities  and  use  of  the  simulator. 

Reset  -  permits  instructor  to  ''return'*  the  simulated  aircraft  to  a 
stored  set  of  conditions  and  parameters. 

Total  System  Freeze  -  permits  instructor  to  interrupt  and  suspend 
simulated  flight  by  freezing  all  system  parameters. 

Partial  Freeze  -  permits  instructor  to  freeze  various  flight 
parameters  or  parameter  combinations  such  as  altitude,  heading, 
position,  attitude,  flight  system,  etc. 

Recorded  Briefing  -  permits  instructor  to  provide  student  with 
information  about  a  structured  training  session  through  audio/visual 
media  presentation. 

Demonstration  -  permits  instructor  to  demonstrate  optimal  electronic 
warfare  procedures  by  prerecording  and  subsequently  playing  back  a 
simulated  engagement. 

Reoord/Playback  -  permits  instructor  to  record  and  subsequently 
playback  a  segment  of  simulated  flight. 

Automated  Malfunction  Insertion  -  permits  instructor  to  pre-program  a 
sequence  of  aircraft  component  malfunctions  and/or  emergency 
conditions. 

Hard  Copy  -  provides  a  record  of  alphanumeric  and/or  graphic 
performance  data  for  debriefing  purposes. 

Programmed  Threat  Control  -  conpu ter -controlled  standardized  training 
sessions  based  on  pre-programmed  event  sequences. 

Manual  Threat  Control  -  permits  instructor  to  modify  threat  scenarios 
during  a  training  session. 

Procedures  Monitoring  -  permits  instructor  to  monitor  discrete  actions 
performed  by  the  student  in  accordance  with  a  procedural!/  defined 
checklist. 

Parameters  Monitoring  -  permits  instructor  to  monitor  various 
instrument  readings,  control  settings,  aircraft  states,  or 
navigational  profiles. 

Electronic  Warfare  Performance  Scoring  -  provides  a  performance  metric 
that  sunmarizes  the  outcomes  of  EW  engagements. 


